RAPID MULTI-SPECTRAL IMAGING METHODS AND APPARATUS AND APPLICATIONS FOR CANCER DETECTION AND LOCALIZATION

• US 20130012794A1
• Filed: 03/17/2011
• Published: 01/10/2013
• Est. Priority Date: 03/17/2010
• Status: Active Grant

• Current Assignee: Haishan Zeng, Yasser Fawzy
• Original Assignee: Haishan Zeng, Yasser Fawzy
• Inventors: Zeng, Haishan, Fawzy, Yasser
• Granted Patent: US 9,654,745 B2
• Time in Patent Office: Days
• US Class Current: 600/328
• CPC Class Codes: A61B 1/000096 using artificial intelligence
• CPC Class Codes: A61B 1/00057 provided with means for testing or calibration
• CPC Class Codes: A61B 1/00186 with imaging filters
• CPC Class Codes: A61B 1/043 for fluorescence imaging
• CPC Class Codes: A61B 1/05 characterised by the image sensor, e.g. camera, being in the distal end portion
• CPC Class Codes: A61B 1/0607 for annular illumination
• CPC Class Codes: A61B 1/063 for monochromatic or narrow-band illumination
• CPC Class Codes: A61B 1/0638 providing two or more wavelengths
• CPC Class Codes: A61B 1/0646 with illumination filters
• CPC Class Codes: A61B 1/0655 Control therefor
• CPC Class Codes: A61B 1/0676 at distal tip of an endoscope
• CPC Class Codes: A61B 1/0684 using light emitting diodes [LED]
• CPC Class Codes: A61B 5/0059 using light, e.g. diagnosis by transillumination, diascopy, fluorescence A61B5/0093, A61B5/1455, A61B5/02416 and A61B5/0261 take precedence
• CPC Class Codes: A61B 5/0071 by measuring fluorescence emission
• CPC Class Codes: A61B 5/0086 using infrared radiation
• CPC Class Codes: G01J 2003/2866 Markers; Calibrating of scan
• CPC Class Codes: G01J 3/10 Arrangements of light sources specially adapted for spectrometry or colorimetry
• CPC Class Codes: G01J 3/28 Investigating the spectrum using colour filters G01J3/51
• CPC Class Codes: G01J 3/36 Investigating two or more bands of a spectrum by separate detectors
• CPC Class Codes: G01N 21/35 using infrared light G01N21/39 takes precedence
• CPC Class Codes: H04N 23/11 : for generating image signals from visible and infrared light wavelengths
• CPC Class Codes: H04N 23/125 : Colour sequential image capture, e.g. using a colour wheel
• CPC Class Codes: H04N 23/555 : for picking-up images in sites, inaccessible due to their dimensions or hazardous conditions, e.g. endoscopes or borescopes
• CPC Class Codes: H04N 9/646 : for image enhancement, e.g. vertical detail restoration, cross-colour elimination, contour correction, chrominance trapping filters

Abstract

Methods and apparatus for video rate or near video rate quantitative imaging of tissue physiological and morphological properties from visible/NIR light spectral images obtain rapid multi-spectral reflectance images by illuminating with a series of spectra containing multiple narrow wavelength bands. An iterative light-transport based inversion algorithm may be applied for correcting the intensity of the spectral images from the geometry/coupling effect as well as from the scattering amplitude distortions. The method can produce video rate absorption as well as scattering spectral images that can be further analyzed very rapidly, using matrix-based rapid inversion algorithms to produce more detailed quantitative images containing information relevant to tissue physiology and morphology.

Claims

1. Apparatus for multispectral imaging of an object, the apparatus comprising:

a light source configurable to emit light having a first spectrum characterized by a first plurality of narrow bands, the apparatus arranged to direct the light from the light source onto an area of the object; and

,

a multi-channel imaging light detector arranged to image the area of the object and configured to provide multi-channel image data; and

an image processing system configured to extract a plurality of spectral images from the multi-channel image data, the plurality of spectral images comprising a spectral image corresponding to each of the plurality of narrow bands.

2. Apparatus according to claim 1 wherein the light source is controllable to vary a spectrum of the light, the apparatus comprises a controller configured to switch the light source to emit light having a second spectrum characterized by a second plurality of narrow bands, the second plurality of narrow bands of the second spectrum occurring at wavelengths different from the first plurality of narrow bands of the first spectrum, and the controller is configured to operate the imaging light detector to obtain a first multi-channel image of the object while the light source is illuminating the area of the object with the light having the first spectrum and to operate the imaging light detector to obtain a second multi-channel image of the object while the light source is illuminating the area of the object with the light having the second spectrum.

3. Apparatus according to claim 1 wherein the light source has a plurality of modes, in each of the modes the light source is configured to emit light having a corresponding spectrum characterized by a corresponding plurality of narrow bands, such that, for each different one of the modes, the plurality of narrow bands occur at a corresponding different set of wavelengths.

4. Apparatus according to claim 3 wherein the apparatus comprises a controller configured to repeat, for each of a plurality of selected modes:

switching the light source to the selected mode; and

operating the detector to obtain corresponding multi-channel image data while the light source is illuminating the area of the object with the light having the spectrum corresponding to the selected mode.

5. Apparatus according to claim 1 wherein the light source comprises a broadband light source and a filter wheel comprising a plurality of filters, each of the plurality of filters having a corresponding filter function comprising a plurality of narrow pass bands.

6. Apparatus according to claim 4 wherein the broadband light source comprises a Xenon arc lamp.

7. Apparatus according to claim 1 wherein the light source comprises a plurality of narrow-band light emitters.

8. Apparatus according to claim 7 wherein the narrow band light emitters comprise semiconductor light emitters.

9. Apparatus according to claim 8 wherein the narrow band light emitters comprise light-emitting diodes.

10. Apparatus according to claim 1 wherein the light source comprises a broadband light source and one or more tunable filters.

11. Apparatus according to claim 10 wherein the tunable filters comprise acousto-optic tunable filters or liquid crystal tunable filters.

12. Apparatus according to claim 1 wherein the light source comprises a digital micromirror device based spectrally programmable light source.

13. Apparatus according to claim 1 wherein the plurality of narrow bands are in that portion of the electromagnetic spectrum consisting of visible and near infrared light.

14. Apparatus according to claim 1 wherein the plurality of narrow bands have wavelengths in the range of 390 to 1000 nm.

15. Apparatus according to claim 14 wherein the plurality of narrow bands have wavelengths in the range of about 400 nm to about 800 nm.

16. Apparatus according to claim 1 wherein the imaging detector comprises a plurality of light-sensing arrays each responsive to light in a corresponding pass band.

17. Apparatus according to claim 16 wherein each of the narrow bands of the plurality of narrow bands falls within a different one of the pass bands.

18. Apparatus according to claim 1 wherein the detector comprises an imaging array comprising an array of light-sensing elements overlaid by a pattern of colour filters.

19. Apparatus according to claim 18 wherein each of the colour filters passes light from two or more of the narrow bands in amounts determined by filter characteristics corresponding to the colour filters.

20. Apparatus according to claim 19 comprising a data store containing calibration information wherein the image processing system is configured to decompose the multi-channel image data using the calibration information to yield a plurality of narrow-band images corresponding to the plurality of narrow bands.

21. Apparatus according to claim 4 wherein the detector comprises an imaging array comprising an array of light-sensing elements overlaid by a pattern of colour filters, each of the colour filters passes light from two or more of the narrow bands in amounts determined by filter characteristics corresponding to the colour filters and the apparatus comprises a data store containing calibration information corresponding to each of the modes and an image processing subsystem configured to decompose the multi-channel image data using the corresponding calibration information to yield a plurality of narrow-band images corresponding to the plurality of narrow bands.

22. Apparatus according to claim 21 wherein the plurality of narrow bands consists of three narrow bands and the three narrow bands include a first narrow band in a blue wavelength range, a second narrow band in a green wavelength range and a third narrow band in a red/near infrared wavelength range.

23. Apparatus according to claim 1 wherein a number of the narrow bands is equal to a number of channels of the multi-channel imaging detector.

24. Apparatus according to claim 1 comprising an image processing device configured to process the plurality of spectral images to yield one or more of corrected albedo spectral images and corrected absorption and corrected scattering spectral images.

25. Apparatus according to claim 24 wherein the image processing device is configured to derive at least one of:

a physiological image of the object and a morphological image of the object based on one or both of the corrected albedo spectral images and the corrected absorption and corrected scattering spectral images.

26. Apparatus according to claim 25, wherein the physiological image comprises at least one of:

an oxygenated hemoglobin concentration image, a de-oxygenated hemoglobin image, a total hemoglobin concentration image and a water volume image.

27. Apparatus according to claim 25 wherein the morphological image is based on at least one of:

a scattering volume image, a mucosa layer thickness image and a scattering particle volume fraction image.

28. Apparatus according to claim 1 wherein the apparatus is configured as an endoscope.

29. Apparatus according to claim 28 wherein the light source is at a proximal end of the endoscope and the endoscope comprises a light guide connected to carry light from the light source to a distal end of the endoscope.

30. Apparatus according to claim 29 wherein the imaging detector is located at the distal end of the endoscope.

31. Apparatus according to claim 28 wherein the light source is located at a distal end of the endoscope.

32. Apparatus according to claim 31 wherein the light source comprises a plurality of light-emitting diodes arranged around an aperture for the imaging detector.

33. Apparatus according to claim 1 wherein the narrow bands have bandwidths of 20 nm or less.

34. Apparatus according to claim 1 wherein the apparatus is configured to obtain the spectral images at a rate of 24 or more sets of spectral images per second.

35. Apparatus according to claim 1 configurable to provide a white light illumination mode wherein the light source illuminates the area with white light and the detector obtains colour images of the area.

36. Apparatus for multispectral imaging of an object, the apparatus comprising:

a light source for illuminating an object, the light source being adapted to generate simultaneously light in a plurality of narrow wavelength bands to produce a plurality of spectral images, each of the narrow wavelength bands containing different wavelengths within a visible/NIR range, wherein the light source is configured to change the wavelengths of the plurality of narrow bands with time to create different illumination spectral profiles;

an image detecting device for receiving the images; and

an image processing device for processing and analyzing the plurality of spectral images and producing corrected albedo spectral images, or corrected absorption and corrected scattering spectral images, and deriving at least one physiological image of the object and at least one morphological image of the object based from the corrected albedo spectral images or the corrected absorption and corrected scattering spectral images.

37. An apparatus according to claim 36 wherein the light source is selected from the group comprising a wide band light source, a programmable digital micro-mirror light source, a laser driven light source and a light emitting diode.

38. Apparatus according to claim 37, wherein the light source comprises a wide band light source which further comprises a rotating filter wheel with a plurality of filters that each pass two or more narrow bands simultaneously, each of the narrow bands being within different wavelength.

39. Apparatus according to claim 37 comprising an endoscope, wherein the light source comprises a plurality of light emitting diodes located at a distal end of the endoscope and the plurality of light emitting diodes are arranged to provide uniform illumination of an area of the object in a plurality of narrow wavelength bands.

40. Apparatus according to claim 36, wherein the different narrow band wavelengths are evenly spaced.

41. Apparatus according to claim 36, wherein the different narrow band wavelengths are unevenly spaced.

42. Apparatus according to claim 36, wherein the plurality of narrow bands include at least one narrow band in a blue wavelength range, at least one narrow band in a green wavelength range and at least one narrow band in a red/near infrared wavelength range.

43. Apparatus according to claim 36, wherein the image detecting device comprises at least three CCDs.

44. Apparatus according to claim 36 comprising an endoscope wherein the image detecting device is located at a distal end of an endoscope.

45. Apparatus according to claim 36, wherein the image detecting device comprises a Bayer filter mosaic color camera, wherein the Bayer filter mosaic camera is calibrated to obtain reflectance imaging signals that correspond to reflectance images for each of the plurality of narrow wavelength bands.

46. Apparatus according to claim 45, comprising a data store containing calibration information comprising a square calibration matrix.

47. Apparatus according to claim 36 comprising an intensity calibration module for obtaining the corrected albedo spectral images or the corrected absorption and corrected scattering spectral images, and a rapid inversion module for quantifying the at least one physiological image and at least one morphological image based on the corrected albedo spectral images or the corrected absorption and corrected scattering spectral images.

48. Apparatus according to claim 47 wherein the intensity calibration module uses measured reflectance spectra from a pixel of an image and in-vitro optical absorption spectra of the object to correct the intensity of the plurality of spectral images.

49. Apparatus according to claim 47, wherein the intensity calibration module defines at least a measurement geometry intensity calibration constant Ka to correct for a coupling angle and measurement distance variation between the object and an imaging means, and an albedo correction intensity calibration constant Ks to correct for a relative intensity nature of measurements.

50. Apparatus according to claim 49 wherein the apparatus is configured to apply the geometry constant Ka to determine the corrected albedo spectral images and the apparatus is configured to apply the albedo correction constant Ks to determine the corrected absorption and corrected scattering spectral images from an albedo image using an in-vitro absorption spectra of the object.

51. Apparatus according to claim 48, wherein the rapid inversion module is based on a linear matrix inversion model.

52. A multi-spectral imaging method comprising:

exposing an area of an object to light having a spectrum comprising a plurality of narrow wavelength bands;

acquiring image data comprising an image of the exposed object using a multi-channel imaging detector; and

extracting a plurality of spectral images from the image data, the plurality of spectral images comprising a spectral image corresponding to each of the plurality of narrow bands.

53. A method according to claim 52 comprising repeating exposing the area of the object and acquiring the image data a plurality of times wherein the method comprises changing the spectrum for different repetitions by changing wavelengths of the bands of the plurality of narrow wavelength bands.

54. A method according to claim 52 wherein the wavelength bands are within a spectral range consisting of visible and near infrared light.

55. A method according to claim 52 comprising repeating the method at a rate of 24 Hz or more.

56. A method according to claim 52 wherein exposing an area of an object to light comprises passing the light through a filter having a plurality of passbands corresponding to the plurality of narrow wavelength bands.

57. A method according to claim 53 wherein exposing an area of an object to light comprises passing the light through a filter having a plurality of passbands corresponding to the plurality of narrow wavelength bands and the method comprises passing the light through a different filter for each of the repetitions.

58. A method according to claim 57 wherein the filters are mounted around a wheel and the method comprises rotating the wheel and coordinating operation of the imaging detector with rotation of the wheel to acquire at least one image using the imaging detector while light is exposing the object through each of the filters.

59. A method according to claim 56 wherein extracting a plurality of spectral images from the image data comprises processing the image data using calibration data wherein the method applies different calibration data for images corresponding to each different one of the filters.

60. A method according to claim 52 comprising synthesizing a white light reflectance image from the spectral images.

61. A method according to claim 52 comprising processing the spectral images to obtain one or more of:

corrected albedo spectral images and corrected absorption and corrected scattering spectral images.

62. A method according to claim 52 comprising processing the spectral images to obtain one or more of:

a physiological image of the object and a morphological image of the object.

63. A method according to claim 62 wherein the object comprises tissue and the physiological image comprises one or more of:

an oxyhemoglobin image, a deoxyhemoglobin image, a tissue water volume image and an image of another chromophore distribution.

64. A multi-spectral imaging method comprising:

exposing an area of an object to light in N narrow wavelength bands and obtaining images of the object while it is exposed to the light;

wherein the exposing is performed in a sequence of steps and each step comprises simultaneously exposing the object to light having a spectrum consisting essentially of a set of n of the N narrow wavelength bands at a time and obtaining an image of the object using a multi-channel imaging detector; and

,

processing multi-channel image data from the multi-channel imaging detector to obtain spectral images corresponding to the N narrow wavelength bands.

65. A method according to claim 64 comprising repeating the method at a video rate and displaying the spectral images or images derived from the spectral images.

66. A method according to claim 65 wherein the video rate is a rate of at least 24 Hz.

67. A method for quantification of tissue physiological and morphological information, the method comprising the steps of:

illuminating an object simultaneously with plurality of narrow wavelength bands of illumination, each of the narrow bands having a different wavelength in a visible/NIR range;

producing a plurality of spectral images;

correcting an intensity of the spectral images using measured reflectance spectra from a pixel of the image and obtaining corrected albedo spectral images or corrected absorption and corrected scattering spectral images; and

deriving at least one physiological image and at least one morphological image from the corrected albedo spectral images or the corrected absorption and corrected scattering spectral images.