Method and system to maximize space-time resolution in a Time-of-Flight (TOF) system

• US 20110292370A1
• Filed: 05/29/2010
• Published: 12/01/2011
• Est. Priority Date: 05/29/2009
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Canesta Incorporated (Microsoft Corporation)
• Inventors: Hills, Gage, Perry, Travis, Bamji, Cyrus
• Granted Patent: US 8,988,661 B2
• Time in Patent Office: Days
• US Class Current: 356/5.01
• CPC Class Codes: G01C 3/08 Use of electric radiation detectors
• CPC Class Codes: G01S 17/36 with phase comparison between the received signal and the contemporaneously transmitted signal
• CPC Class Codes: G01S 17/894 3D imaging with simultaneous measurement of time-of-flight at a 2D array of receiver pixels, e.g. time-of-flight cameras or flash lidar
• CPC Class Codes: G01S 7/4912 Receivers

Abstract

Phase-based TOF systems operate with reduced depth error due to motion blur, and/or spatial blur, and/or pixel offset by intelligently determining how best to combine pixel values, and how best to compensate for individual pixel offsets. Such determination(s) may be carried out on a per pixel basis, dynamically, in real-time during TOF operation, or on archived TOF data. Offsets for individual pixels may be dynamically calculated and subtracted from the values acquired by those pixels Individual pixel offsets may be calculated for example by combining data acquired by the same pixel at two acquisitions, 180° out of phase with respect to each other. Calculated offsets may be averaged, or on a per pixel basis, and if target object motion is detected, one or more offset calculations can be discarded rather than averaged to reduce motion blur. Offsets acquired a priori during a TOF system calibration procedure may be used.

Claims

1. For use with a phase-type TOF system that emits modulated optical energy toward a target object, and detects with an array of pixels a portion of said modulated optical energy reflected by said target, and determines depth Z to said target object by combining pixel values to compare phase of emitted said modulated optical energy with phase of reflected said optical energy, a method of reducing at least one of depth error due to motion of said target object, depth error due to spatial edges of said target object, and depth error due to pixel offset, the method including the following steps:

(a) determining how best to combine values from pixels in said sensor to yield accurate depth data; and

(b) determining how best to compensate for individual pixel offsets to yield accurate depth data.

2. The method of claim 1, wherein at least one of step (a) and step (b) is carried out on a per pixel basis

3. The method of claim 1, wherein at least one of step (a) and step (b) is carried out dynamically.

4. The method of claim 1, wherein at least one of step (a) and step (b) is carried out in real-time.

5. The method of claim 1, wherein step (a) and step (b) are carried out upon previously acquired and stored time-of-flight data.

6. The method of claim 1, wherein step (b) includes dynamically calculating offsets for individual pixels and subtracting dynamically calculated said offsets from pixel values acquired by said pixels.

7. The method of claim 6, wherein individual pixel offsets are calculated by combining data acquired for a same pixel at two acquisitions 180 degrees out of phase with respect to each other.

8. The method of claim 7, wherein said calculated offsets are averaged.

9. The method of claim 8, wherein on a per pixel base, upon detection of motion of said target object, at least one offset calculation is discarded rather than averaged.

10. The method of claim 1, wherein step (b) includes using offsets acquired a priori during a time-of-flight system calibration procedure.

11. For use with a phase-type TOF system that emits modulated optical energy toward a target object, and detects with an array of pixels a portion of said modulated optical energy reflected by said target, and determines depth Z to said target object by combining pixel values to compare phase of emitted said modulated optical energy with phase of reflected said optical energy, a system to reduce at least one depth error due to motion of said target object, depth error due to spatial edges of said target object, and depth error due to pixel offset, the system comprising:

means for determining how best to combine values from pixels in said sensor to yield accurate depth data; and

means for determining how best to compensate for individual pixel offsets to yield accurate depth data.

12. The system of claim 11, wherein at least one of said means for determining how best to combine values and said means for determining how best to compensate function on a per pixel basis

13. The system of claim 11, wherein at least one of said means for determining how best to combine values and said means for determining how best to compensate function dynamically.

14. The system of claim 11, wherein at least one of said means for determining how best to combine values and said means for determining how best to compensate function in real-time.

15. The system of claim 11, wherein at least one of said means for determining how best to combine values and said means for determining how best to compensate operate upon previously acquired and stored time-of-flight data.

16. The system of claim 11, wherein said means for determining how best to compensate compensates by dynamically calculating offsets for individual pixels and subtracting dynamically calculated said offsets from pixel values acquired by said pixels.

17. The system of claim 16, wherein said means for determining how best to compensate calculates individual pixel offsets by combining data acquired for a same pixel at two acquisitions 180 degrees out of phase with respect to each other.

18. The system of claim 17, wherein said means for determining how best to compensate averages said calculated offsets.

19. The system of claim 18, wherein on a per pixel base, upon detection of motion of said target object, at least one offset calculation is discarded rather than averaged.

20. The method of claim 11, wherein said means for determining how best to compensate uses offsets acquired a priori during a time-of-flight system calibration procedure.