DIGITAL LIGHT PROCESSING HYPERSPECTRAL IMAGING APPARATUS

• US 20100056928A1
• Filed: 08/10/2009
• Published: 03/04/2010
• Est. Priority Date: 08/10/2008
• Status: Active Grant

• Current Assignee: Board of Regents of the University of Texas System (University of Texas System)
• Original Assignee: Board of Regents of the University of Texas System (University of Texas System)
• Inventors: FRANCIS, Robert P., LIVINGSTON, Edward, CADEDDU, Jeffrey A., UFRET-VINCENTY, Rafael, ZUZAK, Karel
• Granted Patent: US 8,406,859 B2
• Time in Patent Office: Days
• US Class Current: 600/476
• CPC Class Codes: A61B 3/12 for looking at the eye fundus, e.g. ophthalmoscopes A61B3/13 takes precedence
• CPC Class Codes: A61B 5/0059 using light, e.g. diagnosis by transillumination, diascopy, fluorescence A61B5/0093, A61B5/1455, A61B5/02416 and A61B5/0261 take precedence
• CPC Class Codes: A61B 5/0071 by measuring fluorescence emission
• CPC Class Codes: A61B 5/0075 by spectroscopy, i.e. measuring spectra, e.g. Raman spectroscopy, infrared absorption spectroscopy A61B5/0071 takes precedence
• CPC Class Codes: A61B 5/0084 for introduction into the body, e.g. by catheters A61B5/1459 takes precedence
• CPC Class Codes: A61B 5/0086 using infrared radiation
• CPC Class Codes: A61B 5/14551 for measuring blood gases
• CPC Class Codes: A61B 5/4064 Evaluating the brain A61B5/031, A61B5/369, A61B5/14553 take precedence
• CPC Class Codes: A61B 5/411 Detecting or monitoring allergy or intolerance reactions to an allergenic agent or substance
• CPC Class Codes: A61B 5/413 Monitoring transplanted tissue or organ, e.g. for possible rejection reactions after a transplant
• CPC Class Codes: A61B 5/4842 Monitoring progression or stage of a disease
• CPC Class Codes: G01J 2003/1213 Filters in general, e.g. dichroic, band
• CPC Class Codes: G01J 2003/1282 Spectrum tailoring
• CPC Class Codes: G01J 3/02 Details
• CPC Class Codes: G01J 3/0208 using focussing or collimating elements, e.g. lenses or mirrors; performing aberration correction
• CPC Class Codes: G01J 3/021 using plane or convex mirrors, parallel phase plates, or particular reflectors
• CPC Class Codes: G01J 3/0229 using masks, aperture plates, spatial light modulators or spatial filters, e.g. reflective filters
• CPC Class Codes: G01J 3/0264 Electrical interface; User interface
• CPC Class Codes: G01J 3/10 Arrangements of light sources specially adapted for spectrometry or colorimetry
• CPC Class Codes: G01J 3/2803 using photoelectric array detector
• CPC Class Codes: G01J 3/2823 : Imaging spectrometer
• CPC Class Codes: G01N 21/31 : Investigating relative effect of material at wavelengths characteristic of specific elements or molecules, e.g. atomic absorption spectrometry G01N21/72 takes precedence
• CPC Class Codes: G01N 21/39 : using tunable lasers
• CPC Class Codes: G06T 2207/10036 : Multispectral image; Hyperspectral image

Abstract

A hyperspectral imaging system having an optical path. The system including an illumination source adapted to output a light beam, the light beam illuminating a target, a dispersing element arranged in the optical path and adapted to separate the light beam into a plurality of wavelengths, a digital micromirror array adapted to tune the plurality of wavelengths into a spectrum, an optical device having a detector and adapted to collect the spectrum reflected from the target and arranged in the optical path and a processor operatively connected to and adapted to control at least one of: the illumination source; the dispersing element; the digital micromirror array; the optical device; and, the detector, the processor further adapted to output a hyperspectral image of the target. The dispersing element is arranged between the illumination source and the digital micromirror array, the digital micromirror array is arranged to transmit the spectrum to the target and the optical device is arranged in the optical path after the target.

Claims

1. A hyperspectral imaging system having an optical path, said system comprising:

an illumination source adapted to output a light beam, said light beam illuminating a target;

a dispersing element arranged in said optical path and adapted to separate said light beam into a plurality of wavelengths;

a spatial light modulator adapted to tune said plurality of wavelengths into a spectrum;

an optical device comprising a detector and adapted to collect said spectrum reflected from said target and arranged in said optical path; and

,

a processor operatively connected to and adapted to control at least one of;

said illumination source;

said dispersing element;

said spatial light modulator;

said optical device; and

, said detector, said processor further adapted to output a hyperspectral image of said target,

wherein said dispersing element is arranged between said illumination source and said spatial light modulator, said spatial light modulator is arranged to transmit said spectrum to said target and said optical device is arranged in said optical path after said target.

2. The hyperspectral imaging system according to claim 1 wherein said illumination source is an optically tunable radiation source is selected from the group consisting of:

a supercontinuum laser and a light emitting diode.

3. The hyperspectral imaging system according to claim 1 wherein said illumination source produces said light beam comprising said plurality of wavelengths ranging from 250 nm to 2,500 nm.

4. The hyperspectral imaging system according to claim 1, wherein said target is a fluorescent target.

5. The hyperspectral imaging system according to claim 4, wherein said fluorescent target comprises a natural fluorescence of a tissue or a fluid.

6. The hyperspectral imaging system according to claim 4, wherein said fluorescent target comprises a natural fluorescence, a fluorescent dye, a fluorescence resonance energy donor, a fluorescence resonance energy acceptor, a fluorescence quencher or combinations thereof.

7. The hyperspectral imaging system according to claim 4, wherein said fluorescent target comprises a fluorescence molecule selected from the group consisting of:

indocyanine green;

5-carboxyfluorescein (5-FAM);

6-carboxyfluorescein (6-FAM);

fluorescein-5-isothiocyanate (FITC);

2′

7′

-dimethoxy-4′

5′

-dichloro-6-carboxyfluorescein (JOE);

rhodamine and rhodamine derivatives such as N,N,N′

,N′

-tetramethyl-6-carboxyrhodamine (TAMRA);

6-carboxyrhodamine (R6G);

tetramethyl-indocarbocyanine (Cy3);

tetramethyl-benzindocarbocyanine (Cy3.5);

tetramethyl-indodicarbocyanine (Cy5);

tetramethyl-indotricarbocyanine (Cy7);

6-carboxy-X-rhodamine (ROX);

hexachloro fluorescein (HEX);

tetrachloro fluorescein TET;

R-Phycoerythrin;

4-(4′

-dimethylaminophenylazo) benzoic acid (DABCYL);

5-(2′

-aminoethyl)aminonaphthalene-1-sulfonic acid (EDANS); and

, combinations thereof.

8. The hyperspectral imaging system according to claim 1, wherein said target comprises a structure selected from the group consisting of:

a tissue in a bloodstream;

a tissue in a lymphatic system;

a tissue in a bodily secretion;

a tissue in a biopsy;

a skin tissue;

a tissue of a blood vessel;

a retinal tissue;

a tissue imaged by laparoscopy;

a tissue imaged by endoscopy;

a central nervous system tissue;

a bodily fluid;

an organ; and

, combinations thereof.

9. The hyperspectral imaging system according to claim 1, wherein said dispersing element is selected from the group consisting of:

a grating;

a prism;

a tunable filters;

an electromechanical optical filter wheel;

an acousto-optical tunable filter;

a liquid-crystal tunable filter;

a digital micromirror device;

an electro-optical filter;

a holographic filter; and

, combinations thereof.

10. The hyperspectral imaging system according to claim 1, wherein said spatial light modulator comprises a digital micromirror device.

11. The hyperspectral imaging system according to claim 1, wherein said spectrum comprises a narrow bandwidth of wavelengths of said light beam.

12. The hyperspectral imaging system according to claim 1, wherein said spectrum comprises a complex composition of wavelengths of said light beam and respective portions of said complex composition each comprise an intensity.

13. The hyperspectral imaging system according to claim 1, wherein said optical device comprises a camera.

14. The hyperspectral imaging system according to claim 1, wherein said detector is selected from the group consisting of:

a spectrometer;

a two-dimensional array detector;

a multi-array detector;

an on-chip amplification CCD camera;

a back-illuminated CCD camera;

a liquid nitrogen cooled focal plane array camera;

a photomultiplier tube;

a focal plane array;

a CMOS array and, combination thereof.

15. The hyperspectral imaging system according to claim 1, wherein said detector comprises a plurality of pixels and said processor further comprises a deconvolution algorithm adapted to normalize said spectrum at each of said plurality of pixels prior to processing said hyperspectral image.

16. The hyperspectral imaging system according to claim 1, wherein said processor is adapted to perform at least one of:

tune said spatial light modulator;

trigger said detector for collection of a series of spectroscopic images formatted as a hyperspectral image cube; and

, process said hyperspectral image cube for visualization.

17. The hyperspectral imaging system according to claim 1, wherein said processor comprises an image data acquisition software adapted to perform at least one of:

tune said spatial light modulator;

trigger said detector for collection of a series of spectroscopic images formatted as a hyperspectral image cube; and

, process said hyperspectral image cube for visualization, said processor further comprises a converter to digitize said hyperspectral image cube, wherein said processor is connect to a display.

18. The hyperspectral imaging system according to claim 1, wherein said processor further comprises:

a computer comprising an image data processing software adapted to process digital signals;

a digital signal processor comprising an algorithm adapted for chemometric analysis; and

,

an algorithm library adapted for visualizing chemistry on and within said target for detection, monitoring and diagnosis.

19. The hyperspectral imaging system according to claim 1, wherein said processor is arranged to communicate with a database comprising a library of spectra of fluorescent targets, a library of spectra of absorbent targets, a library of spectra of reflective targets, a library of spectra of organs, a library of spectra of tissues, a library of spectra of physiological substances or combinations thereof.

20. The hyperspectral imaging system according to claim 1, wherein said hyperspectral image is processed at a video rate.

21. The hyperspectral imaging system according to claim 1, wherein said hyperspectral image is formed in less than 1.0 second.

22. The hyperspectral imaging system according to claim 20, wherein said hyperspectral image is formed in approximately 0.33 second.

23. The hyperspectral imaging system according to claim 1 further comprising illumination optics selected from the group consisting of:

a lens;

a microscope;

a dissection microscope;

a surgical microscope;

a slit lamp;

an endoscope;

a laparoscope;

a colonoscope;

funduscope;

broncialscope and, combinations thereof.

24. The hyperspectral imaging system according to claim 1, wherein said hyperspectral imaging system is adapted for non-invasive real-time in vivo or in vitro analysis of said target.

25. The hyperspectral imaging system according to claim 1 further comprising collimating optics adapted to transmit said light beam from said dispersing element to said spatial light modulator as a collimated light beam.

26. The hyperspectral imaging system according to claim 1 further comprising beam shaping optics adapted to transmit said light beam from said spatial light modulator to said target so that said light beam substantially illuminations all of said target.

27. The hyperspectral imaging system according to claim 1, wherein said hyperspectral image comprises chemically encoded data.

28. The hyperspectral imaging system according to claim 27, wherein said chemically encoded data is determined by comparing said spectrum reflected from said target to a known reflected spectrum, by comparing said spectrum reflected from said target to a previously obtained spectrum reflected from said target, or combinations thereof.

29. The hyperspectral imaging system according to claim 1, wherein said target comprises a biomarker adapted to absorb, reflect or fluoresce in response to said spectrum.

30. The hyperspectral imaging system according to claim 29, wherein said biomarker comprises a naturally occurring substance within said target, a substance introduced into said target or combinations thereof.

31. The hyperspectral imaging system according to claim 29, wherein said biomarker comprises water, oxyhemoglobin, deoxyhemoglobin, carboxyhemoglobin, lipid, bilirubin, cholesterol, collagen, glucose, central nervous system fluid proteins, saliva proteins, semen or combinations thereof.

32. A method for obtaining spectral image data from a target in vivo comprising the steps of:

generating a beam of light;

dispersing said light beam with a dispersing element;

separating said dispersed light beam into a spectrum using a spatial light modulator;

illuminating said target with said spectrum, wherein said spectrum subsequently reflects off of said target as a reflected light beam;

collecting said reflected light beam; and

,

directing said collected reflected light beam to a detector adapted to capture spectral image data.

33. The method according to claim 32, further comprising the steps of:

tuning said spatial light modulator;

triggering said detector for collecting said reflected light beam;

forming a hyperspectral image from said spectral image data and,

displaying said hyperspectral image on a display device.

34. The method according to claim 33 wherein said detector comprises a plurality of pixels and said step of forming a hyperspectral image comprises the steps of:

calculating a spectra ratio for each of said plurality of pixels;

filtering said spectra ratios with a data smoothing filter to create a filtered data set;

normalizing said filtered data set to create a normalized data set; and

,

deconvoluting said normalized data set by performing a least squares fit on said normalized data set to create a fitted normalized data set and by scaling said fitted normalized data set to create a gray scale image, wherein said gray scale image is said hyperspectral image.

35. The method according to claim 33 further comprising the steps of:

comparing said hyperspectral image to a library of hyperspectral images to determine if said target matches an image from said library of hyperspectral images.

36. The method according to claim 32, wherein said method involves a human subject and further comprises the step of:

covering said human subject with a NIR transparent material adapted to provide privacy, accessibility and maintain body heat, prior to the step of illuminating said target.

37. A method of obtaining a hyperspectral image of a target comprising the steps of:

generating a light beam;

dispersing said light beam with a dispersing element;

separating said dispersed light beam into a first complex spectrum using a spatial light modulator;

illuminating said target with said first complex spectrum, wherein said first complex spectrum subsequently reflects off of said target as a first reflected light beam;

collecting said first reflected light beam;

directing said collected first reflected light beam to a detector to capture a first spectral image data;

separating said dispersed light beam into a second complex spectrum using said spatial light modulator;

illuminating said target with said second complex spectrum, wherein said second complex spectrum subsequently reflects off of said target as a second reflected light beam;

collecting said second reflected light beam;

directing said collected second reflected light beam to said detector to capture a second spectral image data; and

,

forming said hyperspectral image from said first and second spectral image data.

38. The method according to claim 37, wherein said first and second reflected light beams each comprises reflected, luminescence, fluorescence, autofluorescence, Raman scattered, transmitted, scattered, absorbed, or emitted electromagnetic radiation.

39. The method according to claim 37, wherein said detector comprises:

a processor comprising an image data acquisition software adapted to tune said spatial light modulator, trigger said detector for collection of said first and second spectral image data formatted as a hyperspectral image cube and process said hyperspectral image cube for visualization;

a digital signal process algorithm for analyzing chemometrics of said target; and

,

a display device adapted to display said hyperspectral image.

40. The method according to claim 37, wherein said first and second complex spectrum each comprise a complex composition of wavelengths of said light beam and respective portions of said complex composition each comprise an intensity.

41. The method according to claim 37, wherein said method is adapted for use in combination with a surgical procedure selected from the group consisting of:

a cholecystectomy;

an amputation; and

, a tumor removal.

42. The method according to claim 37, wherein said method is adapted for use in visualizing an organ selected from the group consisting of:

skin, an eye;

a kidney;

a gall bladder;

a liver;

a lung;

a stomach;

a bowel; and

, a brain.

43. The method according to claim 37, wherein said method is adapted for use in visualizing a physiological process selected from the group consisting of:

angiogenesis;

an antigen binding to an antibody;

a drug uptake;

a vascular change;

wound healing;

oxygenation of a retina;

optic nerve oxygenation;

kidney oxygenation.

44. The method according to claim 37, wherein said method is adapted for use in visualizing a disease state selected from the group consisting of:

a burn;

diabetic retinopathy;

cancer progression;

sickle cell;

diabetes;

anemia;

raynauds;

an ulcer;

autoimmune retinitis;

infectious retinitis;

infiltrative neoplastic conditions;

ocular trauma injuries; and

, wound infections.

45. The method according to claim 37, wherein said method is adapted for use in visualizing a physiological substance selected from the group consisting of:

water;

oxyhemoglobin;

deoxyhemoglobin;

carboxyhemoglobin;

lipids;

macular pigments;

retinal photoreceptor pigments;

retinal pigment epithelium;

cholesterol;

glucose;

proteins in central nervous system fluid;

proteins in saliva;

semen; and

, disease biomarkers.

46. A method of monitoring a tissue or an organ viability during periods of ischemia comprising the steps of:

forming a hyperspectral image of said tissue or said organ, wherein said hyperspectral image comprises a quantitative measure of oxygenation of said tissue or said organ;

calculating a length of time of said tissue or said organ viability based on said quantitative measure of oxygenation of said tissue or said organ.

47. The method according to claim 46, wherein said step of forming a hyperspectral image is performed with an apparatus comprising:

an illumination source adapted to output a light beam, said light beam illuminating said tissue or said organ;

a dispersing element arranged in said optical path and adapted to separate said light beam into a plurality of wavelengths;

a spatial light modulator adapted to tune said plurality of wavelengths into a spectrum;

an optical device comprising a detector and adapted to collect said spectrum reflected from said tissue or said organ and arranged in said optical path; and

,

a processor operatively connected to and adapted to control at least one of;

said illumination source;

said dispersing element;

said spatial light modulator;

said optical device; and

, said detector, said processor further adapted to output said hyperspectral image of said tissue or said organ,

wherein said dispersing element is arranged between said illumination source and said spatial light modulator, said spatial light modulator is arranged to transmit said spectrum to said tissue or said organ and said optical device is arranged in said optical path after said tissue or said organ.