Methods and apparatus for imaging of occluded objects from scattered light

• US 9,148,649 B2
• Filed: 10/07/2012
• Issued: 09/29/2015
• Est. Priority Date: 10/07/2011
• Status: Active Grant

• Current Assignee: Massachusetts Institute of Technology
• Original Assignee: Massachusetts Institute of Technology
• Inventors: Raskar, Ramesh, Velten, Andreas
• Primary Examiner(s): Rahaman, Mohammed
• Application Number:
  US13/646,703
• Publication Number:
  US 20130100250A1
• Time in Patent Office: 1,087 Days
• Field of Search: 348/46
• US Class Current: 1/1
• CPC Class Codes: G01S 17/89 for mapping or imaging
• CPC Class Codes: G01S 7/4808 Evaluating distance, position or velocity data
• CPC Class Codes: G01S 7/484 Transmitters
• CPC Class Codes: G01S 7/487 Extracting wanted echo signals , e.g. pulse detection
• CPC Class Codes: H04N 13/254 in combination with electromagnetic radiation sources for illuminating objects

Abstract

In exemplary implementations of this invention, a 3D range camera “looks around a corner” to image a hidden object, using light that has bounced (reflected) off of a diffuse reflector. The camera can recover the 3D structure of the hidden object.

Claims

1. A method of estimating a 3D surface of an object, wherein:

(a) a pulse source emits wave pulses that travel to the object, reflect off of the object, and then travel from the object to a sensor by one or more diffuse reflections;

(b) an occluder prevents the wave pulses from traveling in a direct line from the object to the sensor;

(c) each respective wave pulse has a direction immediately before the wave pulse first diffusely reflects after being emitted by the pulse source;

(d) the direction of the respective wave pulses varies from wave pulse to wave pulse;

(e) the sensor captures streak images while the wave pulses arrive at the sensor after the one or more diffuse reflections;

(f) a processor performs a calculation to estimate the 3D surface, wherein the calculation includes

(i) calculating a voxel grid for a volume in which the object is located,

(ii) for each respective voxel in a set of voxels in the voxel grid, initializing a variable associated with the respective voxel to zero or to a reference value, and

(iii) for each respective pixel of the sensor for each of the respective streak images, incrementing the variable for each voxel that is a candidate, based on time of flight, of being an origin of measured intensity at the respective pixel, the increment being calculated based at least in part on measured intensity at the respective pixel; and

(iv) calculating a second derivative to filter data that results from step (iii).

2. The method of claim 1, wherein the wave pulses comprise electromagnetic radiation.

3. The method of claim 2, wherein the pulse source comprises a laser.

4. The method of claim 3, wherein each wave pulse has a duration, from beginning of pulse to end of pulse, that is less than or equal to one nanosecond.

5. The method of claim 1, wherein the second derivative is taken along a direction in the voxel grid.

6. The method of claim 1, wherein the wave pulses comprise sound waves.

7. The method of claim 1, wherein the calculation to estimate the 3D surface includes a step of performing a local threshold.

8. The method of claim 1, wherein:

(a) each of the wave pulses undergoes a last diffuse reflection, out of the one or more diffusion reflections, before reaching the sensor;

(b) the last diffuse reflection is from a diffusely reflecting surface R;

(c) the pulse source is a laser;

(d) the calculation to estimate the 3D surface includes calibration using a calibration spot that appears in streak images captured by the sensor; and

(e) the calibration spot is created by portion of light, initially from the laser, traveling to the surface R and from the surface R to the sensor.

9. The method of claim 8, wherein the calibration includes correcting image intensity by normalizing based at least in part on calibration spot intensity.

10. The method of claim 8, wherein the calibration includes correcting image time so that the calibration spot is at a correct time position.

11. A method of estimating a 3D surface of an object, wherein:

(a) a pulse source emits wave pulses that travel to the object, reflect off of the object, and then travel from the object to a sensor by one or more diffuse reflections;

(b) an occluder prevents the wave pulses from traveling in a direct line from the object to the sensor;

(c) each respective wave pulse has a direction immediately before the wave pulse first diffusely reflects after being emitted by the pulse source;

(d) the direction of the respective wave pulses varies from wave pulse to wave pulse;

(e) the sensor captures streak images while the wave pulses arrive at the sensor after the one or more diffuse reflections;

(f) a processor performs a calculation to estimate the 3D surface, wherein the calculation includes

(i) calculating a voxel grid for a volume in which the object is located,

(ii) for each respective voxel in a set of voxels in the voxel grid, initializing a variable associated with the respective voxel to zero or to a reference value,

(iii) for each respective pixel of the sensor for each of the respective streak images, incrementing the variable for each voxel that is a candidate of being an origin of measured intensity at the respective pixel, the increment being calculated based at least in part on measured intensity at the respective pixel, and

(iv) filtering data that results from step (iii).

12. The method of claim 11, wherein the wave pulses comprise electromagnetic radiation.

13. The method of claim 12, wherein the pulse source comprises a laser.

14. The method of claim 11, wherein each wave pulse has a duration, from beginning of pulse to end of pulse, that is less than or equal to one nanosecond.

15. The method of claim 11, wherein the filtering comprises taking a second derivative.

16. The method of claim 11, wherein the filtering comprises applying a threshold.

17. The method of claim 11, wherein:

(a) each of the wave pulses undergoes a last diffuse reflection, out of the one or more diffusion reflections, before reaching the sensor;

(b) the last diffuse reflection is from a diffusely reflecting surface R; and

(c) the calculation to estimate the 3D surface includes a step of converting coordinates of the pixels into coordinates of points on the surface R.

18. The method of claim 11, wherein:

(a) after leaving the pulse source, each of the respective wave pulses first diffusely reflects from a point L;

(b) each of the wave pulses undergoes a last diffuse reflection, out of the one or more diffusion reflections, before reaching the sensor;

the last diffuse reflection being from a point w on a diffusely reflecting surface R;

(c) the calculation to estimate the 3D surface includes a step in which allowed values of s are computed and the computed values of s, for a time t for a fixed L and a fixed w, all lie on an ellipsoid with focal points L and w, given by the equation t=r_c+r_l, where S is the 3D surface, s is a point on the surface S, r_c=∥

w−

s∥

, r_l=∥

L−

s∥

, ∥

w−

s∥

is absolute value of time shift by distance from w to s, and ∥

L−

s∥

is absolute value of time shift by distance from L to s.

19. The method of claim 11, wherein:

each of the wave pulses undergo a last diffuse reflection, out of the one or more diffusion reflections, before reaching the sensor;

the last diffuse reflection is from a point w on a diffusely reflecting planar surface R;

S is the 3D surface;

the processor calculates points s on the surface S; and

each of the calculated points s, respectively, contributes a hyperboloid to I_R(u,v,t), where (u,v) are coordinates of w in the plane of the surface R, t is time; and

I_R(u,v,t) is intensity of wave pulse at the surface R, as a function of u, v, and t.

20. An apparatus comprising, in combination:

(a) a pulse source configured to emit wave pulses that travel to an object, reflect off of the object, and then travel from the object to a sensor by one or more diffuse reflections, such that (i) each respective wave pulse has a direction immediately before the wave pulse first diffusely reflects after being emitted by the pulse source, and (ii) the direction of the respective wave pulses varies from wave pulse to wave pulse;

(b) the sensor, wherein the sensor is configured to capture streak images while the wave pulses arrive at the sensor after the one or more diffuse reflections; and

(c) a processor for performing a calculation to estimate the 3D surface, wherein the calculation includes

(i) calculating a voxel grid for a volume in which the object is located,

(ii) for each respective voxel in a set of voxels in the voxel grid, initializing a variable associated with the respective voxel to zero or to a reference value,

(iii) for each respective pixel of the sensor for each of the respective streak images, incrementing the variable for each voxel that is a candidate, based on time of flight, of being an origin of measured intensity at the respective pixel, the increment being calculated based at least in part on measured intensity at the respective pixel, and

(iv) filtering data that results from step (iii).