3D imaging, ranging, and/or tracking using active illumination and point spread function engineering

• US 9,967,541 B2
• Filed: 11/05/2015
• Issued: 05/08/2018
• Est. Priority Date: 11/05/2014
• Status: Active Grant

• Current Assignee: The Board of Regents of the University of Colorado (University of Colorado System)
• Original Assignee: The Board of Regents of the University of Colorado (University of Colorado System)
• Inventors: Piestun, Rafael
• Primary Examiner(s): Czekaj, Dave
• Assistant Examiner(s): Brumfield, Shanika
• Application Number:
  US14/934,031
• Publication Number:
  US 20160125610A1
• Time in Patent Office: 915 Days
• Field of Search: 348 42- 60
• CPC Class Codes: G06T 2200/04 involving 3D image data
• CPC Class Codes: H04N 13/204 using stereoscopic image cameras stereoscopic photography G03B35/00
• CPC Class Codes: H04N 13/254 in combination with electromagnetic radiation sources for illuminating objects

Abstract

Imaging systems and imaging methods are disclosed to estimate a three-dimensional position of an object at a scene and/or generate a three-dimensional image of the scene. The imaging system may include, for example, one or many light sources; an optical system configured to direct light from the one or more light sources into a pattern onto the scene; a mask; a detector array disposed to receive light from the scene through the mask; and at least one processor communicatively coupled with the detector and configured to estimate a depth of a particle within the scene.

Claims

1. An imaging system, comprising:

a pulsed laser;

an optical system configured to direct first light from the pulsed laser into a pattern that illuminates a scene;

a mask that generates a point spread function from second light from the scene that varies based on depth within the scene, wherein the second light from the scene comprises light provided by the scene responsive to illumination by the first light;

a detector array disposed to receive the second light from the scene through the mask and produce an image of the scene, wherein the second light received at the detector array comprises nonlinearly generated harmonic light or multi-photon fluorescence; and

at least one processor communicatively coupled with the detector and configured to estimate a depth of one or more objects within the scene from the image of the scene.

2. The imaging system according to claim 1, further comprising determining a location of the one or more objects within the scene.

3. The imaging system according to claim 1, wherein the imaging system point spread function is optimized to minimize the Cramer Rao lower bound or to maximize the Fisher information with respect to a given parameter such as position localization.

4. The imaging system according to claim 1, wherein the pattern includes a pattern selected from the list consisting of a spot array pattern, a single spot, a striped pattern, a sinusoidal pattern, a pattern of a sparse set of spots, dynamically changing spot pattern, a scanning spot pattern, a pattern of spots that move independently, and a speckle pattern.

5. The imaging system according to claim 1, wherein the processor estimates the depth of the one or more objects based on the representation of the point spread function in the image of the scene.

6. The imaging system according to claim 1, wherein the point spread function comprises a double helix point spread function.

7. The imaging system according to claim 1, wherein the mask includes an optical element selected from the list consisting of an optical element with an extended depth of field, a cubic phase mask, a double helix point spread function mask, diffractive optical element, a grating, a Dammann grating, a diffuser, a phase mask, a hologram, an amplitude mask, a spatial light modulator, and a prism array.

8. The imaging system according to claim 1, wherein the optical system comprises one or more components selected from the group consisting of an active illumination element, phase mask, lens, diffractive optical element, grating, Dammann grating, diffuser, phase mask, hologram, amplitude mask, spatial light modulator, and prism array.

9. A method comprising:

illuminating a scene with first light having a pattern;

directing second light from the scene through an optical system that includes a mask that generates a point spread function that varies based on depth within the scene, wherein the second light comprises light provided by the scene responsive to illumination by the first light;

producing an image of the scene from the second light that passes through the mask using a light detector, the second light received at the light detector and comprising nonlinearly generated harmonic light or multi-photon fluorescence; and

estimating a depth of one or more objects within the scene from the image of the scene.

10. The method according to claim 9, wherein the pattern includes a pattern selected from the list consisting of a spot array pattern, a striped pattern, a sinusoidal pattern, a pattern of a sparse set of spots, dynamically changing spot pattern, a scanning spot pattern, a pattern of spots that move independently, and a speckle pattern.

11. The method according to claim 9, wherein the depth of the one or more objects is estimated based on the representation of the point spread function in the image of the scene.

12. The method according to claim 9, wherein the point spread function comprises a double helix point spread function.

13. The method according to claim 9, wherein the mask includes an optical element selected from the list consisting of an optical element with an extended depth of field, a cubic phase mask, a double helix point spread function mask, diffractive optical element, a grating, a Dammann grating, a diffuser, a phase mask, a hologram, an amplitude mask, a spatial light modulator, and a prism array.

14. A method comprising:

illuminating a scene with first light that has a first light pattern;

generating a first image of the scene from second light from the scene that passes through a first mask using a light detector, wherein the second light comprises light provided by the scene responsive to illumination by the first light, wherein the second light received at the light detector further comprises nonlinearly generated harmonic light or multi-photon fluorescence; and

illuminating the scene with third light that has a second light pattern;

generating a second image of the scene from fourth light from the scene that passes through a second mask using the light detector, wherein the fourth light comprises light provided by the scene responsive to illumination by the third light; and

estimating a depth of one or more objects within the scene from the first image of the scene and the second image of the scene.

15. The method according to claim 14, wherein the first mask and the second mask are the same mask.

16. The method according to claim 14, further comprising generating a three-dimensional image of the scene from the first image of the scene and the second image of the scene.

17. The method according to claim 14, wherein:

the first mask generates a first point spread function from the second light from the scene that varies based on depth within the scene; and

the second mask generates a second point spread function from the fourth light from the scene that varies based on depth within the scene.

18. The method according to claim 17, wherein either or both the first point spread function and the second point spread function comprises a point spread function selected from the list consisting of a double helix point spread function, a helical point spread function, an extended depth of field point spread function, and a cubic phase point spread function.

19. The method according to claim 14, wherein the first pattern includes a pattern selected from the list consisting of a spot array pattern, a striped pattern, a sinusoidal pattern, and a speckle pattern;

and wherein the second pattern includes a pattern selected from the list consisting of a spot array pattern, a striped pattern, a sinusoidal pattern, and a speckle pattern.

20. The method according to claim 14, wherein either or both the first mask and the second mask include an optical element selected from the list consisting of an optical element with an extended depth of field, a cubic phase mask, a double helix point spread function mask, diffractive optical element, a grating, a Dammann grating, a diffuser, a phase mask, a hologram, an amplitude mask, a spatial light modulator, and a prism array.