Superresolution optical fluctuation imaging (SOFI)

• US 9,111,366 B2
• Filed: 09/09/2013
• Issued: 08/18/2015
• Est. Priority Date: 06/02/2009
• Status: Active Grant

• Current Assignee: Sofast GmbH
• Original Assignee: Sofast GmbH
• Inventors: Enderlein, Joerg
• Primary Examiner(s): Vu, Kim
• Assistant Examiner(s): BLOOM, NATHAN J
• Application Number:
  US14/021,553
• Publication Number:
  US 20140099043A1
• Time in Patent Office: 708 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G01N 21/6458 Fluorescence microscopy fluorescence microscopes per se G02B21/0076 and G02B21/16
• CPC Class Codes: G02B 21/0076 arrangements using fluorescence or luminescence
• CPC Class Codes: G02B 21/008 Details of detection or image processing, including general computer control
• CPC Class Codes: G02B 21/367 providing an output produced by processing a plurality of individual source images, e.g. image tiling, montage, composite images, depth sectioning, image comparison
• CPC Class Codes: G02B 27/58 Optics for apodization or superresolution; Optical synthetic aperture systems
• CPC Class Codes: G06T 2207/10056 Microscopic image
• CPC Class Codes: G06T 2207/10064 Fluorescence image
• CPC Class Codes: G06T 3/4053 based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
• CPC Class Codes: H04N 23/951 by using two or more images to influence resolution, frame rate or aspect ratio

Abstract

Statistical analysis techniques based on auto- and cross-correlations/cumulants, of image stacks of fluctuating objects are used to improve resolution beyond the classical diffraction limit and to reduce the background. The time trajectory of every pixel in the image frame is correlated with itself and/or with the time trajectory of an adjacent pixel. The amplitude of these auto- or cross-correlations/cumulants of each pixel, at a given time lag or averaged or integrated over an interval of time lags, is used as the intensity value of that pixel in the generated superresolved optical fluctuation image.

Claims

1. A computer program stored on a non-transitory medium which, when executed by a processor, performs a method for analyzing a field-of-view of independently blinking objects, comprising the following steps:

i) selecting a sample or an observation image comprised of objects from the class of optically signaling objects that stochastically and independently fluctuate;

ii) acquiring a sequence of optical images of the object as a function of time, producing an x, y, t image stack of pixels;

iii) calculating at least one of;

an autocorrelation function, an autocumulant function, a crosscorrelation function, or a crosscumulant function from the temporal fluctuations of selected pixels of the image stack to at least the second order for at least a chosen time lag or a plurality of chosen time lags;

iv) generating a superresolved and background-reduced image including pixels having an intensity value being the amplitude of the function for the at least one chosen time lag or a linear combination of amplitudes of the functions for different chosen time lags.

2. The computer program according to claim 1, wherein the independently blinking objects could be separated by distances shorter than the diffraction limit -resolved distance of the imaging system used.

3. The computer program according to claim 1, wherein the sample objects comprising the sample or the observation image are selected from the class comprising emitting, absorbing or scattering materials and objects labeled with such materials.

4. The computer program according to claim 1, wherein the sequence of optical images is produced from a scanning beam or scan sample, in parallel or in series-wide field, or in pixel-by-pixel sequence.

5. The computer program according to claim 1, wherein the correlation or cumulant functions are of higher order than the second order.

6. The computer program according to claim 5, wherein the acquired sequence of optical images comprises multiple sets of image stacks.

7. A computer program stored on a non-transitory medium which, when executed by a processor, performs a method of superresolution of images of objects being separated by a distance shorter than the optical diffraction limit, comprising the following steps:

introducing observable temporal emission fluctuations in the objects of interest, the fluctuations being stochastic and independent of each other;

generating a magnified image of a field of view encompassing the emitting objects of interest, the magnified image including pixels being substantially smaller than the point spread function introduced in magnification;

sensing temporal fluctuations in the pixels in a succession of image frames corresponding to the temporal fluctuations in the emission of different distributed emitting objects; and

statistically correlating the temporal fluctuations between selected pixels in the image frames; and

generating a superresolved image in at least two dimensions of the emitting objects in the field of view.

8. An electronic device for analyzing a field-of-view of independently blinking objects, comprising:

hardware configured to;

calculate at least one of an autocorrelation function, an autocumulant function, a crosscorrelation function, or a crosscumulant function from temporal fluctuations of selected pixels of an x, y, t image stack to at least the second order for at least a chosen time lag or a plurality of chosen time lags

generating a superresolved and background-reduced image including pixels having an intensity value being the amplitude of the function for the at least one chosen time lag or a linear combination of amplitudes of the functions for different chosen time lags.

9. The electronic device according to claim 8, wherein said ha

select a sample or an observation image rdware device is additionally configured to:

comprised of objects from the class of optically signaling objects that stochastically and independently fluctuate; and

acquire a sequence of optical images of the object as a function of time, producing the x, y, t image stack of pixels.

10. The electronic device according to claim 9, wherein the independently blinking objects could be separated by distances shorter than the diffraction limit-resolved distance of the imaging system used.

11. The electronic device according to claim 9, wherein the sample objects comprising the sample or the observation image are selected from the class comprising emitting, absorbing or scattering materials and objects labeled with such materials.

12. The electronic device according to claim 9, wherein the sequence of optical images is produced from a scanning beam or scan sample, in parallel or in series-wide field, or in pixel-by-pixel sequence.

13. The electronic device according to claim 9, wherein the correlation or cumulant functions are of higher order than the second order.

14. The electronic device according to claim 13, wherein the acquired sequence of optical images comprises multiple sets of image stacks.

15. An electronic device for performing a method of superresolution of images of objects being separated by a distance shorter than the optical diffraction limit, comprising:

hardware configured to;

introduce observable temporal emission fluctuations in the objects of interest, the fluctuations being stochastic and independent of each other;

generate a magnified image of a field of view encompassing the emitting objects of interest, the magnified image including pixels being substantially smaller than the point spread function introduced in magnification;

sense temporal fluctuations in the pixels in a succession of image frames corresponding to the temporal fluctuations in the emission of different distributed emitting objects; and

statistically correlate the temporal fluctuations between selected pixels in the image frames

generating a superresolved image in at least two dimensions of the emitting objects in the field of view.