SYSTEMS, METHODS, AND APPARATUS FOR MULTICHANNEL IMAGING OF FLUORESCENT SOURCES IN REAL TIME

• US 20150182118A1
• Filed: 12/31/2014
• Published: 07/02/2015
• Est. Priority Date: 12/31/2013
• Status: Active Grant

• Current Assignee: Cornell University, Memorial Sloan-Kettering Cancer Center, Quest Medical Imaging BV
• Original Assignee: Cornell University, Memorial Sloan-Kettering Cancer Center, Quest Photonic Devices BV (Olympus Corporation)
• Inventors: Wiesner, Ulrich, Bradbury, Michelle S., Meester, Richard J.C., Patel, Snehal G., Abu-Rustum, Nadeem R., Pauliah, Mohan
• Granted Patent: US 10,986,997 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: A61B 1/00006 of control signals
• CPC Class Codes: A61B 1/000094 extracting biological structures
• CPC Class Codes: A61B 1/000095 for image enhancement
• CPC Class Codes: A61B 1/000096 using artificial intelligence
• CPC Class Codes: A61B 1/0005 combining images e.g. side-by-side, superimposed or tiled
• CPC Class Codes: A61B 1/00096 Optical elements
• CPC Class Codes: A61B 1/00105 characterised by modular construction
• CPC Class Codes: A61B 1/00186 with imaging filters
• CPC Class Codes: A61B 1/00188 with focusing or zooming features
• CPC Class Codes: A61B 1/042 characterised by a proximal camera, e.g. a CCD camera
• CPC Class Codes: A61B 1/043 for fluorescence imaging
• CPC Class Codes: A61B 1/045 Control thereof
• CPC Class Codes: A61B 1/051 Details of CCD assembly
• CPC Class Codes: A61B 1/0646 with illumination filters
• CPC Class Codes: A61B 1/0669 at proximal end of an endoscope
• CPC Class Codes: A61B 1/07 using light-conductive means, e.g. optical fibres
• CPC Class Codes: A61B 1/3132 for laparoscopy
• CPC Class Codes: A61B 5/0071 by measuring fluorescence emission
• CPC Class Codes: A61B 5/0084 for introduction into the body, e.g. by catheters A61B5/1459 takes precedence
• CPC Class Codes: A61B 5/742 using visual displays A61B5/339 takes precedence
• CPC Class Codes: A61B 6/037 : Emission tomography
• CPC Class Codes: A61P 11/00 : Drugs for disorders of the respiratory system
• CPC Class Codes: A61P 17/00 : Drugs for dermatological disorders
• CPC Class Codes: A61P 19/00 : Drugs for skeletal disorders
• CPC Class Codes: A61P 25/00 : Drugs for disorders of the nervous system
• CPC Class Codes: A61P 25/02 : for peripheral neuropathies
• CPC Class Codes: A61P 25/28 : for treating neurodegenerative disorders of the central nervous system, e.g. nootropic agents, cognition enhancers, drugs for treating Alzheimer's disease or other forms of dementia
• CPC Class Codes: A61P 27/02 : Ophthalmic agents
• CPC Class Codes: A61P 29/00 : Non-central analgesic, antipyretic or antiinflammatory agents, e.g. antirheumatic agents; Non-steroidal antiinflammatory drugs [NSAID]
• CPC Class Codes: A61P 3/00 : Drugs for disorders of the metabolism of the blood or the extracellular fluid A61P7/00
• CPC Class Codes: A61P 31/00 : Antiinfectives, i.e. antibiotics, antiseptics, chemotherapeutics
• CPC Class Codes: A61P 35/00 : Antineoplastic agents
• CPC Class Codes: A61P 37/02 : Immunomodulators
• CPC Class Codes: A61P 5/18 : of the parathyroid hormones
• CPC Class Codes: A61P 9/00 : Drugs for disorders of the cardiovascular system

Abstract

Presented herein is a multichannel imaging system capable of detecting and distinguishing multiple fluorescent light sources simultaneously. Also described herein are methods of using the system to image disease or cellular abnormalities, e.g., for diagnostic and/or intraoperative purposes.

Claims

1. A method for optical imaging of a region within a subject, the method comprising:

(a) administering to the subject two or more different probe species each comprising a fluorescent reporter;

(b) directing excitation light into the subject, thereby exciting the fluorescent reporters;

(c) simultaneously detecting fluorescent light of different wavelengths, the detected fluorescent light having been emitted by the fluorescent reporters of the probe species in the subject as a result of excitation by the excitation light so as to discriminate between signals received from each probe species; and

(d) processing signals corresponding to the detected fluorescent light to provide one or more images (e.g. a real-time video stream) of the region within the subject.

2. The method of claim 1, wherein, step (c) is performed without optical time division multiplexing.

3. The method of claim 1, wherein at least one of the probe species comprises nanoparticles.

4. The probe of claim 3, wherein the nanoparticles comprise silica (e.g., and other substances) (e.g., wherein the nanoparticles have a silica architecture and dye-rich core).

5. The probe of claim 4, wherein the dye rich core comprises a fluorescent reporter.

6. The probe of claim 5, wherein the fluorescent reporter is a near infrared or far red dye.

7. The probe of claim 5, wherein the fluorescent reporter is selected from the group consisting of a fluorophore, fluorochrome, dye, pigment, fluorescent transition metal, and fluorescent protein.

8. The probe of claim 5, wherein the fluorescent reporter is selected from the group consisting of Cy5, Cy5.5, Cy2, FITC, TRITC, Cy7, FAM, Cy3, Cy3.5, Texas Red, ROX, HEX, JA133, AlexaFluor 488, AlexaFluor 546, AlexaFluor 633, AlexaFluor 555, AlexaFluor 647, DAPI, TMR, R6G, GFP, enhanced GFP, CFP, ECFP, YFP, Citrine, Venus, YPet, CyPet, AMCA, Spectrum Green, Spectrum Orange, Spectrum Aqua, Lissamine, Europium, Dy800 dye, and LiCor 800 dye.

9. The method of claim 1, wherein, following step (c), a fluorescent reporter is also present in the subject at one or more locations not substantially co-located with another fluorescent reporter.

10. The method of claim 1, wherein the object is an animal (e.g., a human, a mammal, or other animal).

11. The method of claim 1, further comprising any one or both of:

(i) detecting or monitoring a cellular abnormality or disease using the one or more images from the subject; and

(ii) detecting or monitoring normal tissue structures (e.g., marking and discriminating normal tissue structures such as glandular tissues (e.g., parathyroid gland), neural tissues, and/or vascular structures that are present within or lie adjacent to the surgical bed (e.g., are near, or mixed in with, disease or tumor tissue).

12. The method of claim 11, wherein the cellular abnormality or disease comprises at least one member selected from the group consisting of inflammation, cancer, cardiovascular disease, respiratory disease, dermatologic disease, ophthalmic disease, infectious disease, immunologic disease, central nervous system disease, inherited diseases, metabolic diseases, environmental diseases, bone-related disease, neurodegenerative disease, and surgery-related complications.

13. The method of claim 11, wherein the cellular abnormality or disease is sentinel lymph nodes in metastatic melanoma.

14. The method of claim 11, wherein the cellular abnormality or disease is an abnormality of peripheral nerves or nodal disease (e.g., in prostate cancer or in other cancers such as parotid, thyroid, and larynx cancers).

15. The method of claim 11, wherein the cellular abnormality or disease is residual disease in prostate cancer.

23. The method of claim 1, wherein processing signals in step (d) comprises performing, by a processor of a computing device, one or more operations on the signal, the one or more operations selected from the group consisting of scaling, interlacing, chroma resampling, alpha blend mixing, color plane sequencing, frame buffering, test pattern generation, 2D media filtering, color space conversion, control synchronization, and frame reading.

24. The method of claim 1, further comprising:

analyzing, by a processor of a computing device, processed signals based upon information retrieved from a medical imaging data repository (e.g., Nanomed).

25. The method of claim 1, further comprising:

graphically augmenting, by a processor of a computing device, the one or more images (e.g., video streams) using (additional) data retrieved from the medical imaging data repository, wherein graphically augmenting comprises graphically rendering the images with the additional data (e.g., superimposing text or other information from the medical imaging data repository onto the video stream); and

displaying, on a display of a computing device, the one or more graphically augmented images (e.g., graphically augmented video streams).

26. The method of claim 25, wherein the additional data comprises one or more data selected from the group consisting of text (i.e., particle type/composition, ligand, animal model, dose/volume of injectate), optical/PET imaging parameters (i.e., max pixel intensity, % ID/g), camera performance parameters (i.e., gain, exposure time), nodal fluorescence spectral signature (e.g., signal distribution), and histology (e.g., tumor burden).

27. The method of claim 1, further comprising:

visually enhancing, by a processor of a computing device, the one or more images (e.g., video streams); and

displaying, on a display of a computing device, the one or more visually enhanced images.

28. The method of claim 27, wherein visually enhancing the one or more images comprises enhancing the graphical contrast between two or more different fluorescent reporters.

29. The method of claim 1, wherein processing signals in step (d) further comprises performing spectral deconvolution of the images.

30. The method of claim 1, further comprising performing, by the processor, texture-based classification of the images.

16. A portable imaging apparatus, comprising:

a light source configured to deliver multiple excitation wavelengths of light to excite a plurality of different fluorescent reporters that produce fluorescent light at two or more distinguishable wavelengths;

a prism configured to direct light received through a lens onto a plurality of spatially-separated detectors such that said detectors can measure, in real-time, different emitted signals simultaneously; and

a processor configured to process signals corresponding to the detected fluorescent light at the two or more distinguishable wavelengths to provide images of fluorescence of the two or more different fluorescent reporters within a subject.

17. The imaging apparatus of claim 16, wherein the light source comprises two or more lasers and/or a light engine.

18. The imaging apparatus of claim 16, wherein the lens is a single axis optical lens.

19. The imaging apparatus of claim 16, wherein the apparatus further comprises a multi-band filter positioned in front of the lens wherein the multiband filter is configured to block any high power excitation light coming from the light source (and therefore the filter is tuned to the light engine laser light), but will be transparent for all other light (i.e. the visible light and all emission wavelengths of interest).

20. The imaging apparatus of claim 16, wherein the apparatus comprises narrow band filters each positioned between the prism and a respective detector.

21. The imaging apparatus of claim 16, wherein the prism is a dichroic prism.

22. The imaging apparatus of claim 16, wherein the prism comprises at least two surfaces each comprising a different coating.

31. An imaging apparatus, comprising:

optics and a plurality of detectors for simultaneously receiving a plurality of signals, each signal corresponding to a unique fluorescent reporter within a subject (e.g., patient);

a first signal pre-conditioning module for performing a first set of image processing operations on a first signal of the plurality of signals, the first signal corresponding to a first unique reporter (e.g., fluorescent reporter) within the subject;

a second signal pre-conditioning module for performing the first set of image processing operations on a second signal of the plurality of signals, the second signal corresponding to a second unique reporter (e.g., fluorescent reporter) within the subject, wherein the first and second signal conditioning modules are configured to synchronously (e.g., simultaneously) perform image processing on their respective signals (e.g., configured to perform the operations at the same time;

e.g., wherein each signal comprises a video stream and wherein each frame of the video stream is processed by both the first and second signal pre-conditioning device at the same time as each other, followed by the respective next video frames being processed at the same time as each other);

optionally, a third and/or subsequent signal pre-conditioning modules for performing the first set of image processing operations on a third and/or subsequent signal of the plurality of signals, each signal corresponding to a unique reporter; and

a monitor for displaying the processed signals (e.g., the signals may be further processed prior to display).

32. The apparatus of claim 31, wherein each of the first and second signal pre-conditioning modules (and, optionally, the third and/or subsequent signal pre-conditioning modules) is a member selected from the group consisting of a field programmable gate array, an application-specific integrated circuit, and a central processing unit.

33. The apparatus of claim 31, wherein the first and second signal pre-conditioning modules (and, optionally, the third and/or subsequent signal pre-conditioning modules) exist on a single physical device.

34. The apparatus of claim 31, wherein the first set of image processing operations comprises one or more members selected from the group consisting of fast Fourier transformation, discrete Fourier transformation, finite impulse response filtering, and infinite impulse response filtering.

35. The apparatus of claim 31, further comprising:

a first signal post-conditioning module for performing a second set of image processing operations on the first signal;

a second signal post-conditioning module for performing the second set of image processing operations on the second signal, wherein the first and second signal post-conditioning modules are configured to synchronously (e.g., simultaneously) perform image processing on their respective signals (e.g., configured to perform the operations at the same time;

e.g., wherein each signal comprises a video stream and wherein each frame of the video stream is processed by both the first and second signal post-conditioning device at the same time as each other, followed by the respective next video frames being processed at the same time as each other); and

,

optionally, a third and/or subsequent signal post-conditioning module for performing the second set of image processing operations on a third and/or subsequent signal of the plurality of signals, wherein the second set of image processing operations comprises one or more members selected from the group consisting of scaling, interlacing chroma resampling, alpha blend mixing, color plane sequencing, frame buffering, test pattern generation, 2D media filtering, color space conversion, control synchronization, and frame reading.

36. The apparatus of claim 35, wherein each of the first and second signal post-conditioning modules (and, optionally the third and/or subsequent signal post-conditioning module(s)) is a member selected from the group consisting of a field programmable gate array, an application-specific integrated circuit, and a central processing unit.

37. The apparatus of claim 35, wherein the first and second signal post-conditioning modules (and, optionally the third and/or subsequent signal post-conditioning module(s)) exist on a single board unit.

38. The apparatus of claim 31, further comprising a multiplexing module configured to multiplex the first signal and second signal (e.g., as received, as pre-conditioned, or, preferably, as post-conditioned).

39. The apparatus of claim 38, wherein the multiplexing module is additionally configured to multiplex the third and/or subsequent signals.

40. The apparatus of claim 38, comprising a processor configured to retrieve (additional) data from a medical imaging data repository and graphically render the additional data with the multiplexed signals (e.g., superimpose and/or graphically augment the additional data with the multiplexed signals).