Plural detector time-of-flight depth mapping

• US 8,803,952 B2
• Filed: 12/20/2010
• Issued: 08/12/2014
• Est. Priority Date: 12/20/2010
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Katz, Sagi, Adler, Avishai, Yahav, Giora, Tardif, John
• Primary Examiner(s): Perungavoor, Sath V
• Assistant Examiner(s): Le, Peter D
• Application Number:
  US12/973,671
• Publication Number:
  US 20120154542A1
• Time in Patent Office: 1,331 Days
• Field of Search: 348/47
• US Class Current: 348/47
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/235 using a wireless connection, e.g. infrared or piconet
• CPC Class Codes: A63F 13/52 involving aspects of the displayed game scene
• CPC Class Codes: A63F 2300/1031 using a wireless connection, e.g. Bluetooth, infrared connections
• CPC Class Codes: A63F 2300/1087 comprising photodetecting means, e.g. a camera
• CPC Class Codes: A63F 2300/66 for rendering three dimensional images
• CPC Class Codes: G01S 17/87 Combinations of systems using electromagnetic waves other than radio waves
• CPC Class Codes: G01S 17/894 3D imaging with simultaneous measurement of time-of-flight at a 2D array of receiver pixels, e.g. time-of-flight cameras or flash lidar

Abstract

A depth-mapping method comprises exposing first and second detectors oriented along different optical axes to light dispersed from a scene, and furnishing an output responsive to a depth coordinate of a locus of the scene. The output increases with an increasing first amount of light received by the first detector during a first period, and decreases with an increasing second amount of light received by the second detector during a second period different than the first.

Claims

1. A depth-mapping method comprising:

directing modulated infrared or near-infrared illumination onto a scene;

exposing first and second detectors oriented along different optical axes to light dispersed from the scene;

for each of a series of candidate depths into the scene, mapping a pixel of the first detector to 3D coordinates having the candidate depth, reverse mapping the 3D coordinates to a candidate pixel of the second detector, and computing a depth based on an integrated response from the pixel of the first detector versus the candidate pixel of the second detector, the computed depth increasing with an increasing amount of light received at the pixel of the first detector during a first period and decreasing with an increasing amount of light received at the candidate pixel of the second detector during a second period;

for each of the series of candidate depths, assessing agreement between the candidate depth and the corresponding computed depth; and

furnishing an output associated with the pixel of the first detector, the output based on the computed depth of closest agreement to the corresponding candidate depth.

2. The method of claim 1, wherein the scene includes a plurality of loci, and wherein the output is one of a plurality of outputs corresponding to the plurality of loci, each output indicating a depth of its corresponding locus.

3. The method of claim 1, wherein the first detector forms a first image of the scene and the second detector forms a second image of the scene, wherein the first and second images each comprise a rectangular array of pixels, and wherein the first and second periods overlap and differ in duration.

4. The method of claim 1 further comprising:

mapping the pixel of the first detector to 3D coordinates having the computed depth of closest agreement to the corresponding candidate depth;

reverse mapping the 3D coordinates to a refined second pixel of the second detector;

computing a refined depth based on an integrated response from the pixel of the first detector versus the refined second pixel of the second detector; and

furnishing a recomputed output associated with the pixel of the first detector, the recomputed output based on the refined depth.

5. The method of claim 4 further comprising repeating said mapping, reverse mapping, and computing for a finite number of iterations, or until the recomputed output has converged.

6. The method of claim 5 further comprising invalidating the recomputed output if the recomputed output does not converge in the finite number of iterations.

7. The method of claim 1, wherein the output varies substantially linearly with a ratio of the first amount of light to the second amount of light.

8. The method of claim 1 further comprising directing pulsed illumination onto the scene.

9. The method of claim 1 further comprising invalidating the output when the computed depth of closest agreement to the corresponding candidate depth is outside of a predefined range.

10. The method of claim 9, wherein the predefined range is defined based on a depth of a neighboring locus of the scene.

11. The method of claim 3 further comprising aligning the second image to the first image.

12. The method of claim 2, wherein a shallowest locus of the scene is more than one meter from the first detector, and a deepest locus of the scene is less than ten meters from the first detector.

19. The method of claim 1 wherein the light to which the first and second detectors are exposed includes the modulated infrared or near-infrared illumination reflected back from the scene, and wherein wavelength-responsiveness of the first and second detectors is limited substantially to the modulated infrared or near-infrared illumination.

20. The method of claim 1 wherein the candidate pixel of the first detector is mapped to the 3D coordinates using a geometric mapping function of the first detector, and wherein the 3D coordinates are reverse mapped to define a candidate pixel of the second detector using a geometric mapping function of the second detector.

13. A vision system comprising:

a modulated illumination source configured to illuminate a scene using modulated infrared or near-infrared illumination;

first and second detectors oriented along different optical axes and arranged to detect light dispersed from the scene, such light including the modulated infrared or near- infrared illumination reflected back from the scene, with wavelength-responsiveness of the first and second detectors limited substantially to the modulated infrared or near-infrared illumination;

a controller operatively coupled to the first and second detectors and to a source, and configured to;

for each of a series of candidate depths into the scene, map a pixel of the first detector to 3D coordinates having the candidate depth, reverse map the 3D coordinates to a candidate pixel of the second detector, and compute a depth based on an integrated response from the pixel of the first detector versus the candidate pixel of the second detector, the computed depth increasing with an increasing amount of light received at the pixel of the first detector during a first period and decreasing with an increasing amount of light received at the candidate pixel of the second detector during a second period;

for each of the series of candidate depths, assess agreement between the candidate depth and the corresponding computed depth; and

furnish an output associated with the pixel of the first detector, the output based on the computed depth of closest agreement to the corresponding candidate depth.

14. The system of claim 13, wherein the source is pulse-modulated with a pulse-width of fifteen to twenty nanoseconds.

15. The system of claim 13, wherein the source comprises one or more of an infrared or near-infrared light-emitting diode and a laser.

16. The system of claim 13, wherein the first and second detectors are adjacent or separated by two to twenty centimeters.

17. A depth-mapping method comprising:

directing pulsed infrared or near-infrared illumination to a scene, the scene including a plurality of loci;

exposing first and second detectors oriented along different optical axes to light dispersed from the scene, such light including the pulsed infrared or near-infrared illumination reflected back from the scene, with wavelength-responsiveness of the first and second detectors limited substantially to the modulated infrared or near-infrared illumination;

forming a first image of the scene at a first detector and a second image of the scene at a second detector, the first and second images each comprising a rectangular array of pixels;

for each of a series of candidate depths into the scene, mapping a pixel of the first detector to 3D coordinates having the candidate depth, reverse mapping the 3D coordinates to a candidate pixel of the second detector, and computing a depth based on an integrated response from the pixel of the first detector versus the candidate pixel of the second detector, the computed depth increasing with an increasing amount of light received at the pixel of the first detector during a first period and decreasing with an increasing amount of light received at the candidate pixel of the second detector during a second period;

for each of the series of candidate depths, assessing agreement between the candidate depth and a corresponding computed depth; and

furnishing an output associated with the pixel of the first detector, the output based on the computed depth of closest agreement to the corresponding candidate depth.

18. The method of claim 17, wherein the first and second images are formed at least partly from light of the same illumination pulse.