Extended depth of field imaging for high speed object analysis

• US 8,009,189 B2
• Filed: 06/03/2008
• Issued: 08/30/2011
• Est. Priority Date: 01/25/1999
• Status: Expired due to Fees

• Current Assignee: Cytek Biosciences Inc
• Original Assignee: Amnis Corporation (DiaSorin S.p.A.)
• Inventors: Bauer, Richard, Basiji, David, Frost, Keith, Liang, Luchuan, Hall, Brian, Ortyn, William, Perry, David
• Primary Examiner(s): Patel; Haresh N
• Application Number:
  US12/132,394
• Publication Number:
  US 20080234984A1
• Time in Patent Office: 1,183 Days
• Field of Search: 348/80, 348/79, 356/326, 356/73, 356/419, 382/133
• US Class Current: 348/80
• CPC Class Codes: G01N 15/1433 using image recognition
• CPC Class Codes: G01N 15/147 the analysis being performed on a sample stream
• CPC Class Codes: G01N 2015/1452 Adjustment of focus; Alignment
• CPC Class Codes: G01N 21/6458 Fluorescence microscopy fluorescence microscopes per se G02B21/0076 and G02B21/16
• CPC Class Codes: G02B 21/365 Control or image processing arrangements for digital or video microscopes G02B21/361, G02B21/362 take precedence
• CPC Class Codes: G02B 27/0012 Optical design, e.g. procedures, algorithms, optimisation routines
• CPC Class Codes: G02B 27/0075 with means for altering, e.g. increasing, the depth of field or depth of focus

Abstract

A high speed, high-resolution flow imaging system is modified to achieve extended depth of field imaging. An optical distortion element is introduced into the flow imaging system. Light from an object, such as a cell, is distorted by the distortion element, such that a point spread function (PSF) of the imaging system is invariant across an extended depth of field. The distorted light is spectrally dispersed, and the dispersed light is used to simultaneously generate a plurality of images. The images are detected, and image processing is used to enhance the detected images by compensating for the distortion, to achieve extended depth of field images of the object. The post image processing preferably involves de-convolution, and requires knowledge of the PSF of the imaging system, as modified by the optical distortion element.

Claims

1. A method for generating a three-dimensional map of an object, wherein the three-dimensional map exhibits a first extended depth of field along a first axis and a second extended depth of field along a second axis, comprising the steps of:

(a) collecting light from the object along both a first collection path comprising the first axis and a second collection path comprising the second axis, where the first axis and the second axis are not parallel;

(b) deforming an optical wave front of light from the object to produce modified light, such that light in both the first collection path and the second collection path comprises modified light, and a point spread function (PSF) of an imaging system used to collect the modified light is substantially invariant across the first extended depth of field and the second extended depth of field;

(c) focusing modified light from the first collection path to produce a first image of the object, the first image being formed by light simultaneously collected from a first plurality of different focal planes along the first axis, the first plurality of different focal planes comprising the first extended depth of field;

(d) detecting the first image of the object using a first detector to generate first image data;

(e) focusing modified light from the second collection path to produce a second image of the object, the second image being formed by light simultaneously collected from a second plurality of different focal planes along the second axis, the second plurality of different focal planes comprising the second extended depth of field;

(f) detecting the second image of the object using a second detector to generate second image data;

(g) processing the first and second image data to reduce artifacts introduced by deforming the optical wave front, to produce a first corrected extended depth of field image corresponding to the first collection path, and a second corrected extended depth of field image corresponding to the second collection path; and

(h) using the first corrected extended depth of field image and the second corrected extended depth of field image to produce the three-dimensional model of the object.

2. The method of claim 1, further comprising the step of analyzing the three-dimensional model of the object to determine at least one characteristic of the object.

3. The method of claim 1, wherein there is relative motion between the object and the imaging system used to produce the first image of the object and the second image of the object.

4. The method of claim 3, wherein the object is entrained in a flow of fluid.

5. The method of claim 1, wherein at least one of the first extended depth of field and the second extended depth of field comprises about fifteen microns.

6. The method of claim 1, wherein the step of processing the image data comprises the steps of:

(a) determining the PSF of the imaging system configured to deform the optical wave front for the first collection path and the second collection path;

(b) using the PSF function for the first collection path to produce the first corrected extended depth of field image of the object; and

(c) using the PSF function for the second collection path to produce the second corrected extended depth of field image of the object.

7. The method of claim 1, wherein processing the image data comprises the step of de-convolving the first and second image data.

8. The method of claim 7, wherein the step of de-convolving the image data reduces spatial broadening and contrast loss induced by the step of deforming the optical wave front.

9. The method of claim 1, wherein the step of deforming the optical wave front comprises the step of using an optical element to deform the optical wave front.

10. The method of claim 9, wherein the optical element comprises at least one element selected from the group consisting essentially of:

(a) a phase plate configured to induce a phase deviation in the optical wave front;

(b) an optical element configured to induce a spherical aberration in light received from the object; and

(c) a cuvette having different optical thicknesses at different locations, such that imaging through the different locations of the cuvette induces different degrees of wave front deformation.

11. The method of claim 9, wherein the step of using an optical element to deform the optical wave front comprises the steps of:

(a) using a first optical element to modify light for the first collection path; and

(b) using a second optical element to modify light for the second collection path.

12. The method of claim 1, further comprising the step of spectrally dispersing the light in the first collection path before focusing the modified light to produce the first image, such that the first image comprises a spectrally dispersed image.

13. The method of claim 12, further comprising the step of spectrally dispersing the light in the second collection path before focusing the modified light to produce the second image, such that the second image comprises a spectrally dispersed image.

14. The method of claim 1, wherein the step of deforming the optical wave front comprises the step of modifying the aberration correction of an objective lens.

15. A method for generating a stereoscopic view of an object, while there is relative motion between the object and an imaging system used to produce the stereoscopic view, wherein the stereoscopic view exhibits a first extended depth of field along a first axis and a second extended depth of field along a second axis, where the first axis and the second axis are not parallel, comprising the steps of:

(a) using a first optical element to induce an aberration in an optical wave front of light from the object to produce a first modified light along a first collection path, such that a point spread function (PSF) of the imaging system as modified by the first optical element is substantially invariant across the first extended depth of field;

(b) using a second optical element to induce an aberration in an optical wave front of light from the object to produce a second modified light along a second collection path, such that a PSF of the imaging system as modified by the second optical element is substantially invariant across the second extended depth of field;

(c) focusing the first modified light to produce a first image of the object that includes light simultaneously collected from a plurality of different focal planes along the first axis, and focusing the second modified light to produce a second image of the object that includes light simultaneously collected from a plurality of different focal planes along the second axis;

(d) detecting the first image of the object and the second image of the object to generate image data;

(e) processing the image data to reduce artifacts introduced by deforming the optical wave front with the first and second optical elements, to produce a first corrected extended depth of field image of the object and a second corrected extended depth of field image of the object; and

(f) using the first corrected extended depth of field image of the object and the second corrected extended depth of field image of the object to generate the stereoscopic view of the object.

16. The method of claim 15, wherein the step of detecting the first image of the object and the second image of the object comprises the step of using a time delay integration detector to collect each image over a period of time, while the object moves relative to the imaging system.

17. The method of claim 15, further comprising the step of dispersing the first modified light into a plurality of light beams, such that:

(a) the step of focusing the first modified light comprises the step of focusing each of the light beams to produce a respective first image corresponding to that light beam, each respective first image being formed from the first modified light simultaneously collected from the plurality of different focal planes along the first axis;

(b) the step of detecting the first image comprises the step of detecting each respective first image of the object to generate image data; and

(c) the step of processing the image data comprises the step of processing the image data for each respective first image to reduce artifacts introduced by deforming the optical wave front, to produce the first corrected extended depth of field image of the object for each respective first image.

18. The method of claim 17, further comprising the step of analyzing each respective first image to determine at least one characteristic of the object.

19. The method of claim 17, wherein the step of processing the image data comprises the steps of:

(a) determining the PSF of the imaging system as modified by the first optical element configured to deform the optical wave fronts along the first collection path; and

(b) using the PSF function to de-convolve each respective first image, thereby reducing spatial broadening and contrast loss induced by the PSF of the imaging system as modified by the first optical element.

20. The method of claim 15, wherein at least one of the first optical element and the second optical element comprises at least one element selected from the group consisting essentially of:

(a) a phase plate configured to induce a phase deviation in an optical wave front;

(b) an optical element configured to induce a spherical aberration in light from the object; and

(c) a cuvette having different optical thicknesses at different locations, such that imaging through the different locations of the cuvette induces different degrees of wave front deformation.

21. The method of claim 15, wherein the step of deforming the optical wave front comprises the step of modifying the aberration correction of an objective lens.

22. An imaging system adapted to perform extended depth of field imaging of an object, to acquire a first extended depth of field image of the object using light moving away from the object in a first direction, and a second extended depth of field image of the object using light moving away from the object in a second direction, where the first and second directions are different, such that a three-dimensional model of the object can be generated using the first and second extended depth of field images, the system comprising:

(a) a first optical element configured to deform an optical wave front of light moving away from the object in the first direction, and a second optical element configured to deform an optical wave front of light moving away from the object in the second direction, such that a point spread function (PSF) of the imaging system as modified by the first and second optical elements is substantially invariant across a first extended depth of field associated with light moving away from the object in the first direction and a second extended depth of field associated with light moving away from the object in the second direction;

(b) a first collection lens disposed so that light moving away from the object in the first direction passes through the first collection lens and travels along a first collection path, and a second collection lens disposed so that light moving away from the object in the second direction passes through the second collection lens and travels along a second collection path;

(c) a first imaging lens disposed to receive light travelling along the first collection path that has been deformed by the first optical element, thus producing a first deformed extended depth of field image;

(d) a second imaging lens disposed to receive light travelling along the second collection path that has been deformed by the second optical element, thus producing a second deformed extended depth of field image;

(e) a first detector disposed to receive the first deformed extended depth of field image produced by the first imaging lens, producing an output signal that is indicative of the first deformed extended depth of field image, and a second detector disposed to receive the second deformed extended depth of field image produced by the second imaging lens, producing an output signal that is indicative of the second deformed extended depth of field image; and

(f) a processor configured to;

(i) manipulate the output signal from the first detector to reduce artifacts introduced by the first optical element to deform the optical wave front of light from the object in the first collection path, to produce the first extended depth of field image of the object; and

(ii) manipulate the output signal from the second detector to reduce artifacts introduced by the second optical element to deform the optical wave front of light from the object in the second collection path, to produce the second extended depth of field image of the object.

23. The system of claim 22, wherein at least one of the first optical element and the second optical element comprises one element selected from the group consisting essentially of:

(a) a phase plate configured to induce a phase deviation in the optical wave front;

(b) an optical element configured to induce a spherical aberration in light from the object; and

(c) a cuvette having different optical thicknesses at different locations, such that imaging through the different locations of the cuvette induces different degrees of wave front deformation.

24. The system of claim 22, wherein at least one of the first optical element and the second optical element configured to deform the optical wave front of light moving away from the object comprises an adjustable objective lens, such that the aberration of the objective lens can be modified to deform the optical wave front of light from the object.

25. The system of claim 22, wherein at least one of the first optical element and the second optical element is selectively positionable, such that in a first position, that optical element deforms the optical wave front, thereby enabling extended depth of field imaging, while in a second position, that optical element does not deform the optical wave front, thereby enabling non-extended depth of field imaging.

26. The system of claim 22, wherein the processor is configured to use the PSF as modified by the first optical element and the second optical element to de-convolve the first and second output signals, thereby reducing spatial broadening and contrast loss induced by the PSF as modified by each optical element.

27. The system of claim 22, wherein the processor is configured to use the first extended depth of field image and the second extended depth of field image to generate the three-dimensional map of the object.

28. The system of claim 22, further comprising a first dispersion element configured to disperse light traveling along the first collection path before the first imaging lens produces the first deformed extended depth of field image, such that the first deformed extended depth of field image comprises one of:

(a) a single dispersed image; and

(b) a plurality of individual dispersed images.

29. The system of claim 28, further comprising a second dispersion element configured to disperse light traveling along the second collection path before the second imaging lens produces the second deformed extended depth of field image, such that the second deformed extended depth of field image comprises one of:

(a) a single dispersed image; and

(b) a plurality of individual dispersed images.

30. An imaging system adapted to perform extended depth of field imaging of an object in order to produce a stereoscopic image of the object, while there is relative movement between the object and the imaging system, comprising:

(a) a first optical element configured to deform an optical wave front of light moving away from the object in a first direction, thereby producing first modified light, and a second optical element configured to deform an optical wave front of light moving away from the object in a second direction, such that a point spread function (PSF) of the imaging system as modified by each optical element is substantially invariant across a first extended depth of field associated with light moving away from the object in the first direction and a second extended depth of field associated with light moving away from the object in the second direction;

(b) a first collection lens disposed so that light moving away from the object in the first direction passes through the first collection lens and travels along a first collection path, and a second collection lens disposed so that light moving away from the object in the second direction passes through the second collection lens and travels along a second collection path;

(c) a first dispersing component disposed in the first collection path so as to receive the light that has passed through the first collection lens, dispersing the light into a plurality of separate light beams, each light beam being directed away from the first dispersing component in a different predetermined direction, and a second dispersing component disposed in the second collection path so as to receive the light that has passed through the second collection lens, dispersing the light into a plurality of separate light beams, each light beam being directed away from the second dispersing component in a different predetermined direction;

(d) a first imaging lens disposed to receive the light beams from the first dispersing component, producing a plurality of images, each image corresponding to one of the light beams and being projected by the first imaging lens toward a different predetermined location, and a second imaging lens disposed to receive the light beams from the second dispersing component, producing a plurality of images, each image corresponding to one of the light beams and being projected by the second imaging lens toward a different predetermined location;

(e) a first detector disposed to receive the plurality of images produced by the first imaging lens, producing a first output signal that is indicative of imaging light moving away from the object in the first direction at a plurality of different focal planes, and a second detector disposed to receive the plurality of images produced by the second imaging lens, producing a second output signal that is indicative of imaging light moving away from the object in the second direction at a plurality of different focal planes; and

(f) a processor configured to manipulate the first output signal and the second output signal to reduce artifacts introduced by the first optical element and the second optical element configured to deform the optical wave fronts of light from the object, to produce a plurality of first corrected extended depth of field images of the object for light moving away from the object in the first direction, and a plurality of second corrected extended depth of field images of the object for light moving away from the object in the second direction, and to use the plurality of first corrected extended depth of field images and the plurality of second corrected extended depth of field images to produce the stereoscopic image of the object.

31. The system of claim 30, wherein at least one of the first optical element and the second optical element is a phase plate configured to induce a phase deviation in the optical wave front.

32. The system of claim 31, wherein the phase plate is disposed proximate a numerical objective of the imaging system.

33. The system of claim 30, wherein at least one of the first optical element and the second optical element is configured to induce a spherical aberration in light from the object.

34. The system of claim 30, wherein at least one of the first optical element and the second optical element is a cuvette having different optical thicknesses at different locations, such that imaging through the different locations of the cuvette induces different degrees of wave front deformation.

35. The system of claim 30, wherein at least one of the first optical element and the second optical element is selectively positionable, such that in a first position, that optical element deforms the optical wave front, thereby enabling extended depth of field imaging, while in a second position, that optical element does not deform the optical wave front, thereby enabling non-extended depth of field imaging.