High-Throughput Hyperspectral Imaging With Superior Resolution And Optical Sectioning

• US 20160202178A1
• Filed: 01/12/2015
• Published: 07/14/2016
• Est. Priority Date: 01/12/2015
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Verily Life Sciences LLC (Alphabet Inc.)
• Inventors: Backer, Adam, Piponi, Daniele Paolo, Acosta, Victor Marcel, Perreault, John D.
• Granted Patent: US 9,395,293 B1
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G01J 2003/1278 Mask with spectral selection
• CPC Class Codes: G01J 2003/468 of objects containing fluorescent agent
• CPC Class Codes: G01J 3/0205 Optical elements not provided otherwise, e.g. optical manifolds, diffusers, windows
• CPC Class Codes: G01J 3/021 using plane or convex mirrors, parallel phase plates, or particular reflectors
• CPC Class Codes: G01J 3/0229 using masks, aperture plates, spatial light modulators or spatial filters, e.g. reflective filters
• CPC Class Codes: G01J 3/0237 Adjustable, e.g. focussing
• CPC Class Codes: G01J 3/18 using diffraction elements, e.g. grating gratings per se G02B
• CPC Class Codes: G01J 3/502 using a dispersive element, e.g. grating, prism
• CPC Class Codes: G01N 21/255 Details, e.g. use of specially adapted sources, lighting or optical systems
• CPC Class Codes: G01N 21/27 using photo-electric detection G01N21/31 takes precedence; circuits for computing concentration logarithmic circuits G06G7/24; photometric circuits in general G01J
• CPC Class Codes: G01N 2201/06113 Coherent sources; lasers
• CPC Class Codes: G01N 2201/0675 SLM
• CPC Class Codes: G02B 21/0064 multi-spectral or wavelength-selective arrangements, e.g. wavelength fan-out, chromatic profiling G02B21/0076 takes precedence
• CPC Class Codes: G02B 21/06 Means for illuminating specimens
• CPC Class Codes: G02B 21/361 Optical details, e.g. image relay to the camera or image sensor G02B21/364 takes precedence; illumination details G02B21/06 and subgroups
• CPC Class Codes: G02B 26/0808 by means of one or more diffracting elements
• CPC Class Codes: G02B 5/1828 having means for producing variable diffraction controlling the direction of light by means of one or more diffracting elements G02B26/0808; acousto-optical elements G02F1/11, G02F1/33; electro- or magneto-optical diffraction G02F1/292, G02F1/2955
• CPC Class Codes: G02B 6/2931 Diffractive element operating in reflection
• CPC Class Codes: G02F 1/133553 Reflecting elements associated to illuminating devices G02F1/133605
• CPC Class Codes: G02F 1/23 for the control of the colour G02F1/03 - G02F1/21 take precedence
• CPC Class Codes: G02F 1/292 : by controlled diffraction or phased-array beam steering controlled diffraction for optical switching G02F1/31
• CPC Class Codes: G06T 2207/10024 : Color image
• CPC Class Codes: G06T 2207/10056 : Microscopic image
• CPC Class Codes: G06T 2207/10064 : Fluorescence image
• CPC Class Codes: G06T 2207/30004 : Biomedical image processing
• CPC Class Codes: G06T 7/11 : Region-based segmentation
• CPC Class Codes: G06T 7/90 : Determination of colour characteristics
• CPC Class Codes: H04N 23/11 : for generating image signals from visible and infrared light wavelengths
• CPC Class Codes: H04N 23/56 : provided with illuminating means

Abstract

An imaging system includes a light source configured to illuminate a target and a camera configured to image light responsively emitted from the target and reflected from a spatial light modulator (SLM). The imaging system is configured to generate high-resolution, hyperspectral images of the target. The SLM includes a refractive layer that is chromatically dispersive and that has a refractive index that is controllable. The refractive index of the refractive layer can be controlled to vary according to a gradient such that light reflected from the SLM is chromatically dispersed and spectrographic information about the target can be captured using the camera. Such a system could be operated confocally, e.g., by incorporating a micromirror device configured to control a spatial pattern of illumination of the target and to modulate the transmission of light from the target to the camera via the SLM according to a corresponding spatial pattern.

Claims

1. A system comprising:

a light source;

a first camera, wherein the first camera comprises a plurality of light-sensitive elements disposed on a focal surface of the first camera;

a spatial light modulator, wherein the spatial light modulator comprises a reflective layer disposed beneath a refractive layer, wherein the refractive layer is configured to have a refractive index that varies spatially across the spatial light modulator according to a controllable gradient, wherein at least the direction and magnitude of the controllable gradient are electronically controllable, and wherein the refractive layer is chromatically dispersive; and

an optical system, wherein the optical system is configured to (i) direct light from the light source to a target, (ii) direct light emitted from the target in response to the light from the light source toward the spatial light modulator, and (iii) direct light emitted from the target and reflected from the spatial light modulator to the first camera such that the focal surface of the first camera is conjugate to a focal surface passing through the target.

2. The system of claim 1, wherein the refractive layer has a refractive index that varies substantially linearly with wavelength for wavelengths within a specified range of wavelengths.

3. The system of claim 1, wherein the target contains a fluorophore, and wherein the beam of illumination contains light at an excitation wavelength of the fluorophore.

4. The system of claim 1, wherein the optical system is configured to collimate the light emitted from the target in response to illumination that is directed toward the spatial light modulator.

5. The system of claim 1, further comprising:

a micromirror device, wherein the micromirror device comprises a substantially planar array of actuatable mirrors disposed on a surface, wherein respective angles of the actuatable mirrors relative to the surface are electronically controllable,

wherein the optical system is configured to direct the light from the light source to the target via reflection from a first set of one or more of the actuatable mirrors, and wherein the optical system is configured to direct the light emitted from the target in response toward the spatial light modulator via reflection from the first set of one or more actuatable mirrors such that the surface of the micromirror device is conjugate to the focal surface passing through the target, and

wherein the one or more actuatable mirrors in the first set have a first angle relative to the surface of the micromirror device.

6. The system of claim 1, further comprising:

a second camera, wherein the second camera comprises a plurality of light-sensitive elements disposed on a focal surface of the second camera;

a micromirror device, wherein the micromirror device comprises a substantially planar array of actuatable mirrors disposed on a surface, wherein respective angles of the actuatable mirrors relative to the surface are electronically controllable,

wherein the optical system is configured to direct the light from the light source to the target via reflection from a first set of one or more of the actuatable mirrors, wherein the optical system is configured to direct the light emitted from the target in response toward the second camera via reflection from the first set of one or more actuatable mirrors such that the surface of the micromirror device is conjugate to the focal surface passing through the target and such that the focal surface of the second camera is conjugate to a focal surface passing through the target, and wherein the optical system is configured to direct the light emitted from the target in response toward the spatial light modulator via reflection from a second set of one or more of the actuatable mirrors, and

wherein the one or more actuatable mirrors in the first set have a first angle relative to the surface of the micromirror device and the one or more actuatable mirrors in the second set have a second angle relative to the surface of the micromirror device that is different from the first angle.

7. The system of claim 1, further comprising an actuated stage, wherein the actuated stage is configured to control the location of the target relative to the optical system.

8. The system of claim 1, wherein the spatial light modulator comprises an array of cells having respective electronically controllable refractive indexes.

9. A method comprising:

illuminating, by a light source, a target, via an optical system configured to direct light from the light source to the target;

electronically controlling a spatial light modulator during a first period of time such that a refractive layer of the spatial light modulator has a refractive index that varies spatially across the spatial light modulator according to a controllable gradient, wherein the controllable gradient has at least a first specified direction and a first specified magnitude, wherein the spatial light modulator further comprises a reflective layer disposed beneath the refractive layer, and wherein the refractive layer is chromatically disperse;

imaging light emitted from the target in response to the light from the light source during the first period of time using a first camera to produce a first image of the target, wherein the first camera comprises a plurality of light-sensitive elements disposed on a focal surface of the first camera, wherein the optical system is further configured to direct light emitted from the target in response to the light from the light source toward the spatial light modulator and direct light emitted from the target and reflected from the spatial light modulator to the first camera such that the focal surface of the first camera is conjugate to a focal surface passing through the target; and

determining spectrographic information for a particular region of the target based at least on the first image of the target.

10. The method of claim 9, further comprising:

electronically controlling the spatial light modulator during a plurality of further periods of time such that the refractive index of the refractive layer varies spatially across the spatial light modulator according to respective controllable gradients having at least respective specified directions and respective specified magnitudes; and

imaging light emitted from the target in response to the light from the light source during the plurality of further periods of time using the first camera to produce respective further images of the target, wherein determining spectrographic information for a particular region of the target comprises determining spectrographic information based on the first image and the plurality of further images of the target.

11. The method of claim 9, further comprising:

controlling a spectral resolution of the spectrographic information for the particular region of the target by controlling the first specified magnitude of the controllable gradient.

12. The method of claim 11, wherein the target contains a fluorophore, wherein a property of an emission spectrum of the fluorophore is related to a property of the target, wherein controlling a spectral resolution of the spectrographic information comprises controlling the spectrographic resolution to be sufficiently high to determine the property of the target based on determined spectrographic information for the particular region of the target.

13. The method of claim 11, wherein the target contains two fluorophores, wherein the two fluorophores have respective different emission spectra, wherein controlling a spectral resolution of the spectrographic information comprises controlling the spectrographic resolution to be sufficiently high to determine whether the particular region of the target contains the first fluorophore or the second fluorophore based on determined spectrographic information for the particular region of the target.

14. The method of claim 9, wherein the target contains a fluorophore, and wherein illuminating the target comprises emitting light at an excitation wavelength of the fluorophore.

15. The method of claim 9, further comprising:

operating a micromirror device to electronically control respective angles of actuatable mirrors of the micromirror device relative to a surface during the first period of time, wherein the actuatable mirrors comprise a substantially planar array and are disposed on the surface, and wherein operating the micromirror device to electronically control respective angles of actuatable mirrors of the micromirror device comprises controlling a first set of one or more of the actuatable mirrors to have a first angle relative to the surface of the micromirror device during a specified period of time during the first period of time, and

wherein the optical system is configured to direct the light from the light source to the target via reflection from the first set of one or more actuatable mirrors, and wherein the optical system is configured to direct the light emitted from the target in response toward the spatial light modulator via reflection from the first set of one or more actuatable mirrors such that the surface of the micromirror device is conjugate to the focal surface passing through the target.

16. The method of claim 15, wherein operating the micromirror device to electronically control respective angles of actuatable mirrors of the micromirror device relative to a surface comprises controlling the first set of at least one of the actuatable mirrors to have the first angle relative to the surface of the micromirror device during the first period of time, and wherein determining spectrographic information for a particular region of the target based at least on the first image of the target comprises determining spectrographic information for a portion of the target corresponding to the first set of one or more actuatable mirrors based on information in the first image detected by light-sensitive elements of the first camera located proximate to a portion of the focal surface of the first camera corresponding to the portion of the target.

17. The method of claim 9, further comprising:

operating a micromirror device to electronically control respective angles of actuatable mirrors of the micromirror device relative to a surface during the first period of time, wherein the actuatable mirrors comprise a substantially planar array and are disposed on the surface, and wherein operating the micromirror device to electronically control respective angles of actuatable mirrors of the micromirror device comprises controlling a first set of one or more of the actuatable mirrors to have a first angle relative to the surface during a specified period of time during the first period of time and controlling a second set of one or more of the actuatable mirrors to have a second angle relative to the surface during a specified period of time during the first period of time;

imaging light emitted from the target in response to the light from the light source during the first period of time using a second camera to produce a second image of the target, wherein the second camera comprises a plurality of light-sensitive elements disposed on a focal surface of the second camera, wherein the optical system is further configured to direct light emitted from the target in response to the light from the light source to the second camera such that the focal surface of the second camera is conjugate to the focal surface passing through the target; and

wherein the optical system is configured to direct the light from the light source to the target via reflection from the first set of one or more actuatable mirrors, wherein the optical system is configured to direct the light emitted from the target in response toward the second camera via reflection from the first set of one or more actuatable mirrors and toward the spatial light modulator via reflection from the second set of one or more actuatable mirrors such that the surface of the micromirror device is conjugate to the focal surface passing through the target.

18. The method of claim 17, wherein operating the micromirror device to control respective angles of actuatable mirrors of the micromirror device during the first period of time comprises controlling, during respective specified periods of time during the first period of time:

(i) a plurality of respective first sets of one or more actuatable mirrors to have the first angle relative to the surface and (ii) a plurality of respective second sets of one or more actuatable mirrors to have the second angle relative to the surface, and further comprising;

electronically controlling the spatial light modulator during a plurality of further periods of time such that the refractive index of the refractive layer varies spatially across the spatial light modulator according to respective controllable gradients having at least respective specified directions and respective specified magnitudes;

imaging light emitted from the target in response to the light from the light source during the plurality of further periods of time using the second camera to produce respective further images of the target; and

imaging light emitted from the target in response to light from the light source during the plurality of further periods of time using the first camera to produce respective further images of the target, wherein determining spectrographic information for a particular region of the target comprises determining spectrographic information based on the first image and the plurality of further images of the target generated by the first camera.

19. The method of claim 9, further comprising:

controlling, using an actuated stage, the location of the target relative to the optical system.

20. The method of claim 9, wherein the spatial light modulator comprises an array of cells having respective electronically controllable refractive indexes, and wherein electronically controlling a spatial light modulator during a first period of time such that a refractive layer of the spatial light modulator has a refractive index that varies spatially across the spatial light modulator according to a controllable gradient comprises electronically controlling the refractive indexes of the cells such that refractive indexes of the cells vary in a direction corresponding to the first specified direction and at a spatial rate of change corresponding to the first specified magnitude.