Methods and systems for three dimensional optical imaging, sensing, particle localization and manipulation

• US 9,560,338 B2
• Filed: 09/19/2013
• Issued: 01/31/2017
• Est. Priority Date: 04/09/2010
• Status: Active Grant

• Current Assignee: The Board of Regents of the University of Colorado (University of Colorado System)
• Original Assignee: The Board of Regents of the University of Colorado (University of Colorado System)
• Inventors: Piestun, Rafael, Quirin, Sean Albert
• Primary Examiner(s): Li, Ruiping
• Application Number:
  US14/031,887
• Publication Number:
  US 20140015935A1
• Time in Patent Office: 1,230 Days
• Field of Search: 382/154, 382274-275, 382/254, 382/209, 382/106
• US Class Current: 1/1
• CPC Class Codes: G06T 2207/10056 Microscopic image
• CPC Class Codes: G06T 2207/10064 Fluorescence image
• CPC Class Codes: G06T 7/77 using statistical methods
• CPC Class Codes: H04N 13/204 using stereoscopic image cameras stereoscopic photography G03B35/00

Abstract

Embodiments include methods, systems, and/or devices that may be used to image, obtain three-dimensional information from a scence, and/or locate multiple small particles and/or objects in three dimensions. A point spread function (PSF) with a predefined three dimensional shape may be implemented to obtain high Fisher information in 3D. The PSF may be generated via a phase mask, an amplitude mask, a hologram, or a diffractive optical element. The small particles may be imaged using the 3D PSF. The images may be used to find the precise location of the object using an estimation algorithm such as maximum likelihood estimation (MLE), expectation maximization, or Bayesian methods, for example. Calibration measurements can be used to improve the theoretical model of the optical system. Fiduciary particles/targets can also be used to compensate for drift and other type of movement of the sample relative to the detector.

Claims

1. An optical system comprising:

an imaging system that generates a point spread function by control of one or more associated pupil functions of the optical system, the point spread function having a transverse profile that retains its fundamental shape with axial propagation and remains offset from an axis;

a sensor array configured to receive light from at least one object through the imaging system and provide one or more images of the object; and

a processing device configured to;

recover a representation of the one or more associated pupil functions of the imaging system using a phase retrieval algorithm;

determine the position of two concentrations of recorded light in the one or more images that are point spread function representations of the object imaged through the imaging system; and

localize the object in three dimensions based at least in part on a transverse shift of the two concentrations of recorded light in at least one of the one or more images of the object.

2. The optical system of claim 1, wherein one or both of an offset from the axis or a corresponding rate of change of rotation of defocus in the one or more images can change as a spot spins around the axis.

3. The optical system of claim 1, wherein a maximum of the point spread function describes a curve in three-dimensional space similar to a helix with varying offset from a helix axis.

4. The optical system of claim 1, wherein a maximum of the point spread function describes a curve in three-dimensional space similar to a helix with varying pitch.

5. The optical system of claim 1, wherein the imaging system uses optical encoding and/or digital decoding comprising one or more of a deconvolution algorithm or estimates using a Maximum Likelihood Estimation, Bayesian estimator, or expectation maximization algorithm.

6. The optical system of claim 1, wherein the imaging system provides an extended depth of field with a point spread function that presents a single spot that shifts at every transverse plane along a certain range.

7. The optical system of claim 1, wherein the imaging system further comprises at least one of a spatial light modulator, a phase mask, an amplitude mask, and a phase/amplitude mask located in a plane between an object and the sensor array.

22. The optical system of claim 1, wherein the point spread function maximizes 3D Fisher information.

23. The optical system of claim 1, wherein localizing the at least one object in three dimensions includes minimizing a Cramer-Rao bound of the one or more images.

24. The optical system of claim 1, wherein the point spread function describes a curve in three-dimensional space with varying offset from the axis.

8. A method of localizing an object in three dimensions, the method comprising:

illuminating the object with an illumination source;

generating a point spread function by control of one or more associated pupil functions of an optical system;

imaging light from the object modified by the point spread function producing a point spread image;

determining the position of two concentrations of the imaged light in the one or more images that are point spread function representations of the object;

determining a transverse position of at least the positions of the two concentrations of the imaged light; and

estimating the axial location of the object from a transverse position using an estimator that is based on the pupil function of the imaging system.

9. The method according to claim 8, wherein the double-helix point spread function manifest as a pair of lobes that rotate relative to the object'"'"'s axially position.

10. The method according to claim 8, further comprising:

imaging a fiduciary; and

determining at least one of the transverse position, rotation, size, and shape of the object relative to the fiduciary.

11. The method according to claim 8, wherein the double-helix point spread function includes high three-dimensional Fisher Information.

12. The method according to claim 8, wherein the double-helix image comprises a pattern that includes at least one intensity peak which spins around an axis relative to the object'"'"'s axially position.

13. The method according to claim 8, wherein the double-helix point spread function is generated via at least one of a spatial light modulator, a phase mask, an amplitude mask, a hologram, a diffractive optical element, and a spatial light modulator.

14. An optical system comprising:

an imaging system that generates a 3D point spread functions by control of one or more associated pupil functions of the imaging system, wherein the spread function has a transverse profile that retains its fundamental shape with axial propagation and remains offset from an axis;

a sensor array configured to detect light from at least one object through the imaging system and produce one or more images of the object having a representation of the transverse profile of the object; and

a processing device coupled with the imaging system and the sensor array, wherein the processing module is configured to;

receive the one or more images of the object;

determine the position of two concentrations of recorded light in the one or more images that are point spread function representations of the object imaged through the imaging system; and

localize the object in three dimensions based at least in part on a transverse shift of the two concentrations of recorded light in at least one of the one or more images of the object using an estimator that is based on the pupil function of the imaging system.

15. The optical system according to claim 14, wherein the point spread function is a double helix point spread function.

16. The optical system according to claim 14, wherein one or both of an offset from the axis or a corresponding rate of rotation of defocus can change as a spot in the image spins around the axis.

17. The optical system of claim 14, where a maximum of the point spread function describes a curve in three-dimensional space similar to a helix with varying offset from a helix axis.

18. The optical system of claim 14, where a maximum of the point spread function describes a curve in three-dimensional space similar to a helix with varying offset from a pitch.

19. The optical system of claim 14, where a maximum of the point spread function describes a curve in three-dimensional space similar to a helix with varying offset from a helix axis and/or a pitch, and when both the axis and offset are constant, the point spread function describes a helical curve.

20. The optical system of claim 14, wherein the imaging system uses optical encoding and/or digital decoding comprising one or more of a deconvolution algorithm or estimates using a Maximum Likelihood Estimation, Bayesian estimator, or expectation maximization algorithm.

21. The optical system of claim 14, wherein the imaging system further comprises at least one of a spatial light modulator, a phase mask, an amplitude mask, and a phase/amplitude mask located in a plane between an object and the sensor array.