3-D localization and imaging of dense arrays of particles

• US 9,858,464 B2
• Filed: 03/14/2014
• Issued: 01/02/2018
• Est. Priority Date: 03/15/2013
• Status: Active Grant

• Current Assignee: The Board of Regents of the University of Colorado (University of Colorado System)
• Original Assignee: The Board of Regents of the University of Colorado (University of Colorado System)
• Inventors: Piestun, Rafael, Barsic, Anthony
• Primary Examiner(s): Carter, Aaron W
• Application Number:
  US14/777,361
• Publication Number:
  US 20160048963A1
• Time in Patent Office: 1,390 Days
• CPC Class Codes: G01N 21/6458 Fluorescence microscopy fluorescence microscopes per se G02B21/0076 and G02B21/16
• CPC Class Codes: G02B 21/14 affording illumination for phase-contrast observation
• CPC Class Codes: G02B 21/361 Optical details, e.g. image relay to the camera or image sensor G02B21/364 takes precedence; illumination details G02B21/06 and subgroups
• CPC Class Codes: G02B 21/367 providing an output produced by processing a plurality of individual source images, e.g. image tiling, montage, composite images, depth sectioning, image comparison
• CPC Class Codes: G06F 18/00 Pattern recognition
• CPC Class Codes: G06T 2200/04 involving 3D image data
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10056 Microscopic image
• CPC Class Codes: G06T 2207/10064 Fluorescence image
• CPC Class Codes: G06T 2207/30242 Counting objects in image
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06V 30/242 Division of the character sequences into groups prior to recognition; Selection of dictionaries
• CPC Class Codes: H05K 999/99 dummy group

Abstract

Systems, methods, and computer program products are disclosed to localize and/or image a dense array of particles. In some embodiments, a plurality of particles may be imaged using an imaging device. A plurality of point spread function dictionary coefficients of the image may be estimated using a point spread function dictionary; where the point spread function dictionary can include a plurality of spread function responses corresponding to different particle positions. From the point spread function dictionary coefficients the number of particles in the image can be determined. Moreover location of each particle in the image can be determined from the point spread function dictionary coefficients.

Claims

1. A method for detecting the number and/or location of particles, the method comprising:

imaging a plurality of particles using an imaging system having a point spread function, wherein the imaging returns an image;

estimating a plurality of point spread function dictionary coefficients of the image using a point spread function dictionary, wherein the point spread function dictionary includes a plurality of point spread function responses that each correspond with a different particle position; and

determining the number and/or location of the particles in the image from the point spread function dictionary coefficients.

2. The method according to claim 1, further comprising:

imaging the plurality of particles using a second imaging system having a second point spread function, wherein the imaging returns a second image;

estimating a plurality of second point spread function dictionary coefficients of the image using a second point spread function dictionary, wherein the second point spread function dictionary includes a plurality of second point spread function responses that each correspond with a different particle position; and

determining the number and/or location of the particles in the image from the second point spread function dictionary coefficients.

3. The method according to claim 1, further comprising determining the brightness of the particles in the image.

4. The method according to claim 1, wherein the point spread function changes as a function of the axial location of the plurality of particles.

5. The method according to claim 1, further comprising determining a two-dimensional location of each particle in the image from the point spread function dictionary coefficients.

6. The method according to claim 1, further comprising determining a three-dimensional location of the particles in the image from the point spread function dictionary coefficients.

7. The method according to claim 1, wherein the number of particles in the image is determined from a number of point spread function dictionary coefficients above or below a threshold value.

8. The method according to claim 1, wherein the point spread function dictionary comprises a plurality of point spread function responses that correspond with a plurality of three-dimensional spatial position of the particle.

9. The method according to claim 1, wherein the point spread function dictionary comprises a plurality of point spread function responses that correspond with a plurality of two-dimensional spatial position of the particle.

10. The method according to claim 1, wherein the point spread function comprises a double helix point spread function.

11. The method according to claim 1, wherein the point spread function comprises an astigmatic point spread function.

12. The method according to claim 1, wherein the point spread function produces lobes in the image that are displaced based on the axial position of the plurality of particles.

13. The method according to claim 1, wherein the plurality of coefficients of the image of the particles are estimated using either or both a Matching Pursuit algorithm and a Convex Optimization algorithm.

14. The method according to claim 1, wherein the plurality of coefficients of the image of the particles are estimated using an optimization algorithm.

15. An imaging system comprising:

a plurality of optical elements configured to image a plurality of objects, wherein the optical elements generate a point spread function, and wherein the point spread function changes as a function of the axial location of the plurality of objects;

an imager configured to image the plurality of objects located onto an image plane;

a memory including a point spread function dictionary that includes a plurality of point spread function responses that each correspond with a different position of an object; and

a processor configured to estimate a plurality of point spread function dictionary coefficients of an image of the plurality of objects using the point spread function dictionary and determine the number and/or position of objects in the image using the point spread function dictionary coefficients.

16. The system according to claim 15, wherein the processor is further configured to determine a two-dimensional location of the plurality of particles in the image from the point spread function dictionary coefficients.

17. The system according to claim 15, wherein the number of objects in the image is determined from a number of point spread function dictionary coefficients above or below a threshold value.

18. The system according to claim 15, wherein the point spread function produces lobes in the image that are displaced based on the axial position of the plurality of objects.

19. The system according to claim 15, wherein the point spread function dictionary comprises a plurality of point spread function responses that correspond with a three-dimensional spatial position of the objects.

20. The system according to claim 15, wherein the point spread function dictionary comprises a plurality of point spread function responses that correspond with a two-dimensional spatial position of the objects.

21. The system according to claim 15, wherein the plurality of point spread function dictionary coefficients of the image of the objects are estimated based on an optimization algorithm selected from the list consisting of Convex Optimization;

Matching Pursuit;

orthogonal Matching Pursuit;

maximum likelihood estimation;

Bayesian information criteria, maximum-likelihood principle;

maximum-posterior estimator principle;

maximum entropy principle; and

minimum discrimination.

22. A method for detecting particles comprising:

imaging a plurality of particles using a first point spread function that changes as a function of the axial location of a particle, wherein the imaging returns an image;

estimating a first plurality of point spread function dictionary coefficients of the image using a Matching Pursuit algorithm with a point spread function dictionary that includes a plurality of point spread function responses corresponding to a different particle position;

determining a first region within which the particles are located in the image from the first plurality of point spread function dictionary coefficients;

estimating a second plurality of point spread function dictionary coefficients of the image using a Convex Optimization algorithm with a second point spread function dictionary that includes a plurality of point spread function responses corresponding to different particle position; and

determining a location of the particles in the image from the second plurality of coefficients.

23. The method according to claim 22, wherein the number of point spread function responses in the second point spread function dictionary is greater than the number of point spread function responses in the first point spread function dictionary.

24. The method according to claim 22, wherein the spatial resolution of the point spread function responses in the second point spread function dictionary is greater than the spatial resolution of the point spread function responses in the first point spread function dictionary.