Methods and systems for three dimensional optical imaging, sensing, particle localization and manipulation

• US 8,620,065 B2
• Filed: 04/11/2011
• Issued: 12/31/2013
• Est. Priority Date: 04/09/2010
• Status: Active Grant

• Current Assignee: The Board of Regents of the University of Colorado (University of Colorado System)
• Original Assignee: The Board of Regents of the University of Colorado (University of Colorado System)
• Inventors: Piestun, Rafael, Quirin, Sean Albert
• Primary Examiner(s): Li, Ruiping
• Application Number:
  US13/084,345
• Publication Number:
  US 20110249866A1
• Time in Patent Office: 995 Days
• Field of Search: 382/154, 382274-275, 382/254, 382/209, 382/106, 356/4.01, 359/368
• US Class Current: 382/154
• CPC Class Codes: G06T 2207/10056 Microscopic image
• CPC Class Codes: G06T 2207/10064 Fluorescence image
• CPC Class Codes: G06T 7/77 using statistical methods
• CPC Class Codes: H04N 13/204 using stereoscopic image cameras stereoscopic photography G03B35/00

Abstract

Embodiments include methods, systems, and/or devices that may be used to image, obtain three-dimensional information from a scence, and/or locate multiple small particles and/or objects in three dimensions. A point spread function (PSF) with a predefined three dimensional shape may be implemented to obtain high Fisher information in 3D. The PSF may be generated via a phase mask, an amplitude mask, a hologram, or a diffractive optical element. The small particles may be imaged using the 3D PSF. The images may be used to find the precise location of the object using an estimation algorithm such as maximum likelihood estimation (MLE), expectation maximization, or Bayesian methods, for example. Calibration measurements can be used to improve the theoretical model of the optical system. Fiduciary particles/targets can also be used to compensate for drift and other type of movement of the sample relative to the detector.

Claims

1. A method for locating an object in three-dimensions with an optical system, the method comprising:

generating one or more 3D point spread functions (PSF) by control of an associated pupil function of the optical system;

generating one or more experimental images of the object using the one or more 3D PSF of the optical system; and

generating a 3D position of the object by increasing a probability of matching the experimental image to the PSF in the presence of noise either with a maximum likelihood estimator or with a phase retrieval (PR) enabled maximum likelihood estimator.

7. The method of claim 1, wherein an initial guess of coordinates of the object location comprises using a spatial correlation measure.

10. The method of claim 1, wherein generating the one or more experimental images includes collecting using a detector array and digitizing the one or more experimental images.

11. The method of claim 1 wherein at least one of the one or more 3D PSF includes a double helix PSF or a spinning PSF.

12. The method of claim 1, wherein the optical system comprises a spatial light modulator, a phase mask, an amplitude mask, or a phase/amplitude mask located in a Fourier plane or other plane.

15. The method of claim 1, where a maximum of the PSF describes a curve in three-dimensional space similar to a helix but with varying offset from the axis and pitch and when both the axis and offset are constant, the PSF describes a helical curve.

16. The method of claim 1, wherein an imaging system uses optical encoding and digital decoding comprising one or more of a deconvolution algorithm or estimates using a Maximum Likelihood Estimation (MLE), Bayesian estimator, or expectation maximization algorithm.

17. The method of claim 1, wherein the PSF and an imaging system provide an extended depth of field.

18. The method of claim 1, further comprising utilizing a spatial light modulator, a phase mask, an amplitude mask, or a phase/amplitude mask located in a Fourier plane or other plane.

20. The method of claim 1, wherein a transverse profile of at least one of the one or more PSF retains its fundamental shape with axial propagation and remains offset from an arbitrary axis.

21. The method of claim 1, wherein generating a 3D position of the object further comprises estimating the 3D position of the object by increasing a probability of matching the one or more experimental images to the one or more 3D PSF in the presence of noise.

2. A method for locating an object in three-dimensions with an optical system, the method comprising:

generating one or more 3D point spread functions (PSF);

generating one or more experimental images of the object using the one or more 3D PSF of the optical system;

taking a set of calibration images separated by an axial distance;

generating the one or more 3D PSF from the set of calibration images by numerically propagating the set of calibration images with defocus;

searching for a location of the object in an axial direction by numerically propagating the one or more experimental images with an axial shift to generate an axial position estimate and by transversely shifting the axial position estimate to correlate with the one or more experimental images to find transverse position estimates; and

generating a 3D position of the object by increasing a probability of matching the experimental image to the PSF in the presence of noise.

3. The method of claim 2, wherein generating a 3D position of the object further comprises estimating the 3D position of the object by increasing a probability of matching the one or more experimental images to the one or more 3D PSF in the presence of noise.

4. The method of claim 2, further comprising:

solving for the complex field of the set of calibration images offset by the axial distance;

generating an initial phase estimate for the set of calibration images in a first plane;

estimating a first adjacent field by propagating the set of calibration images by the axial distance according to a wave equation governing evolution of a point spread function;

reverse propagating the first adjacent field to generate a revised estimate of the complex field of the set of calibration images.

5. The method of claim 2, further comprising enforcing a set of amplitude constraints, wherein at least one of the one or more 3D PSF include a double helix PSF or a spinning PSF.

6. The method of claim 2, wherein a transverse profile of at least one of the one or more PSF retains its fundamental shape with axial propagation and remains offset from an arbitrary axis.

8. A method for locating an object in three-dimensions with an optical system, the method comprising:

generating one or more 3D point spread functions (PSF);

generating one or more experimental images of the object using the one or more 3D PSF of the optical system; and

generating a 3D position of the object by increasing a probability of matching the experimental image to the PSF in the presence of noise;

wherein a set of initial coordinates is generated by searching for the object location in the axial plane by propagating the experimental image with an axial shift to generate an axial position estimate and transversely cross correlating the axial position estimate with the experimental image to find the transverse position estimates.

9. A method for locating an object in three-dimensions with an optical system, the method comprising:

generating one or more 3D point spread functions (PSF);

generating one or more experimental images of the object using the one or more 3D PSF of the optical system;

generating a 3D position of the object by increasing a probability of matching the experimental image to the PSF in the presence of noise;

wherein generating the 3D position of the object by increasing a probability of matching the experimental image to the PSF in the presence of noise includes a phase retrieval (PR) enabled maximum likelihood estimator (MLE) that models the noise by Poisson, Gaussian, or a mixture distribution.

13. A ranging system, comprising:

a calibration module arranged to take a set of calibration images separated by an axial distance;

an imaging module arranged to take an experimental image of an object, wherein the imaging module is arranged to adjust a focus of the axial ranging system by an axial shift;

a search module arranged to search for an object location within an axial plane by propagating a representation of the experimental image by an axial shift to generate an axial position estimate and by transversely shifting the axial position estimate with the experimental image to find transverse position estimates; and

an estimation module coupled to the calibration module, the imaging module, and the search module, wherein the estimation module is configured to receive the set of calibration images and the experimental image and generate an estimate of a position of the object using a maximum likelihood estimator.

14. The ranging system of claim 13, wherein a propagation module is configured to use multiple experimental images to estimate an original three dimensional function using depth-from-defocus methods with one or more of local diffusion operations, non-local diffusion operations or expectation maximization algorithms.

19. A method for locating an object in three-dimensions with an optical system, the method comprising:

generating one or more 3D point spread functions (PSF) by control of one or more associated pupil functions of the optical system;

generating one or more experimental images of the object using the one or more 3D PSF of the optical system;

generating a model of the one or more 3D PSF and a model of noise;

generating a 3D position of the object by increasing a probability of matching the experimental image to the PSF in the presence of the noise.