System architecture design for time-of-flight system having reduced differential pixel size, and time-of-flight systems so designed

• US 8,203,699 B2
• Filed: 06/26/2009
• Issued: 06/19/2012
• Est. Priority Date: 06/30/2008
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Bamji, Cyrus, Mehta, Swati
• Primary Examiner(s): Issing, Gregory C
• Assistant Examiner(s): Bedard, Antoine J
• Application Number:
  US12/459,160
• Publication Number:
  US 20110304841A1
• Time in Patent Office: 1,089 Days
• Field of Search: 356/5.01, 356/4.01
• US Class Current: 356/5.01
• CPC Class Codes: G01C 3/08 Use of electric radiation detectors
• CPC Class Codes: G01S 17/36 with phase comparison between the received signal and the contemporaneously transmitted signal
• CPC Class Codes: G01S 17/894 3D imaging with simultaneous measurement of time-of-flight at a 2D array of receiver pixels, e.g. time-of-flight cameras or flash lidar
• CPC Class Codes: G01S 7/491 Details of non-pulse systems
• CPC Class Codes: G01S 7/4914 of detector arrays, e.g. charge-transfer gates

Abstract

Embodiments of the present invention provide methods to produce a high performance, feature rich TOF system, phase-based or otherwise using small TOF pixels, single-ended or preferably differential, as well as TOF systems so designed. IC chip area required for pixels is reduced by intelligently off-loading or removing from within the pixel certain components and/or functionality. In some embodiments during a single TOF system capture period, analog values from each pixel are repeatedly sampled and converted to digital values, which are combined and manipulated on the sensor chip. Combining this plurality of values enables appropriately compact data from the sensor chip. Embodiments of the present invention implement a TOF system with high ambient light resilience, high dynamic range, low motion blur and dealiasing support, while advantageously reducing pixel area size relative to prior art TOF pixels.

Claims

1. A method performed by an electronic processor to enhance performance of a time-of-flight (TOF) system that emits optical energy and detects with an array of pixels, each pixel occupying less than 20 μ

m×

20 μ

m in pixel area on an IC substrate, at least a fraction of said optical energy reflected by a target object a distance Z away, the method including the following steps;

(a) digitizing analog detection data obtained from at least some of said pixels during a first operating regime during which at least some of said pixels detect reflected emitted said optical energy during a first integration time interval T1, said digitizing analog detection data obtained from at least some of said pixels during a first operating regime includes acquiring at said TOF system analog detection data during said first operating regime;

(b) digitizing analog detection data obtained from at least some of said pixels during a second operating regime during which at least some of said pixels detect reflected emitted said optical energy during a second integration time interval T2, said digitizing analog detection data obtained from at least some of said pixels during a second operating regime includes acquiring at said TOF system analog detection data during said second operating regime;

(c) combining said digitized analog detection data obtained during said first operating regime and during said second operating regime, wherein relative to a different TOF system that uses digitized data during a single detection integration time interval T1+T2, detection performance of said TOF system is enhanced in at least one aspect selected from the group consisting of detection dynamic differential dynamic range, motion blur, 0°

-180°

offset cancellation, and aliasing interval; and

(d) outputting three-dimensional information based on said combining, wherein operation of said TOF system is determined by a sequence of symbols, said symbols having at least one characteristic selected from a group consisting of (i) at least one symbol in said sequence is associated with a set of pixel waveforms to be applied to a pixel when said symbol is interpreted to be present, and (ii) at least some said symbols and at least some said waveforms are at least one of modifiable and creatable upon download to said TOF of further parameters.

2. The method of claim 1, wherein step (a) and step (b) include acquiring at said TOF system digitized data at a start of said first operating regime and at a start of said second operating regime, wherein detection performance of said TOF system is enhanced in that at least partial cancellation of kT/C noise is achieved;

and

further including multiplying by a non-integer value a digitized data value acquired at a start of at least said first operating regime, wherein kT/C noise is further reduced.

3. The method of claim 1, wherein a digitized data value obtained at an end of said first operating regime is used as a digitized data value at a beginning of said second operating regime.

4. The method of claim 1, wherein step (a) and step (b) have at least one characteristic selected from a group consisting of (i) said time interval T1 is less than 4 ms and said time interval T2 is less than 4 ms, and (ii) digitizing occurs at a rate at least four faster than an input output (TO) data rate for said IC substrate.

5. The method of claim 1, wherein said IC substrate includes digital bitlines and clock lines, and further including switching said bitlines and said clock lines to reduce effects of noise introduced by digitizing at step (a) and at step (b), wherein a signal-to-noise ratio is enhanced.

6. An enhanced performance time-of-flight (TOF) system that emits optical energy and detects with an array of pixels at least a fraction of said optical energy reflected by a target object a distance Z away, the TOF system including:

a plurality of pixels, formed on said array, at least some of said pixels having a pixel area occupying less than 20 μ

m×

20 μ

m in pixel area on an IC substrate that includes said array;

first means for digitizing analog detection data obtained from at least some of said pixels during a first operating regime during which at least some of said pixels detect reflected emitted said optical energy during a first integration time interval T1;

second means for digitizing analog detection data obtained from at least some of said pixels during a second operating regime during which at least some of said pixels detect reflected emitted said optical energy during a second integration time interval T2;

means for combining a digital output from said first means for digitizing and a digital output from said second means for digitizing, wherein relative to a TOF system that uses digitized data during a single detection integration time interval T1+T2, detection performance of said TOF system is enhanced in at least one aspect selected from detection dynamic differential dynamic range, motion blur, 0°

-180°

offset cancellation, and aliasing interval, wherein said IC substrate includes digital bitlines and clock lines, and further including means for switching said bit lines and said clock lines to reduce effects of noise introduced by digitizing data, wherein a signal-to-noise ratio is enhanced.

7. The TOF system of claim 6, further including means for governing operation of said TOF system according to a sequence of symbols, wherein at least one symbol in said sequence is associated with a set of pixel waveforms to be applied to a pixel when said symbol is interpreted to be present.

8. The TOF system of claim 6, wherein said time interval T1 is less than 4 ms and said time interval T2 is less than 4 ms.

9. The TOF system of claim 6, wherein said first means for digitizing and said second means for digitizing each digitize a rate at least four faster than an input output (TO) data rate for said IC substrate.

10. A method performed by an electronic processor for detecting motion utilizing a time-of-flight (TOF) system, comprising:

performing a sequence of operations, said sequence of operations includes a first operation and a second operation, said first operation includes digitizing a first set of analog data acquired from one or more pixel detectors during a first operating regime, said second operation includes digitizing a second set of analog data acquired from said one or more pixel detectors during a second operating regime, said sequence of operations structured to prevent a saturation of said one or more pixel detectors by changing a direction of accumulation during said sequence of operations;

combining said digitized first set of analog data and said digitized second set of analog data in such a way that said combining implements at least one of reducing motion blur, performing 0°

-180°

offset cancellation, or dealiasing; and

outputting three-dimensional information based on said combining.

11. The method of claim 10, wherein:

said second operating regime is performed subsequent to said first operating regime; and

said one or more pixel detectors are implemented as a first part of an integrated circuit, said sequence of operations is performed by circuitry implemented outside said first part.

12. The method of claim 10, wherein:

said first operating regime is associated with a first light phase and said second operating regime is associated with a second light phase different from said first light phase.

13. The method of claim 10, wherein:

said first operating regime is associated with a first emitted light frequency and said second operating regime is associated with a second emitted light frequency different from said first emitted light frequency.

14. The method of claim 10, wherein:

said TOF system is a phase-based TOF system.

15. The method of claim 10, wherein:

said sequence of operations is performed within a frame period.

16. The method of claim 10, wherein:

said sequence of operations includes a common mode reset operation.

17. The method of claim 10, wherein:

said one or more pixel detectors are implemented as a first part of an integrated circuit, said integrated circuit includes bit lines associated with said one or more pixel detectors, a physical layout of said bit lines within said first part includes swapped bit lines.

18. The method of claim 10, wherein:

said one or more pixel detectors are implemented as a first part of an integrated circuit, said integrated circuit includes clock lines in communication with said one or more pixel detectors, a physical layout of said clock lines within said first part includes swapped clock lines.

19. The method of claim 10, wherein:

said performing a sequence of operations includes performing double correlated sampling.

20. The method of claim 10, wherein:

said one or more pixel detectors comprise one or more differential detectors.