CMOS-compatible three-dimensional image sensor IC

• US 6,323,942 B1
• Filed: 09/22/1999
• Issued: 11/27/2001
• Est. Priority Date: 04/30/1999
• Status: Expired due to Term

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Canesta Incorporated (Microsoft Corporation)
• Inventors: Bamji, Cyrus
• Primary Examiner(s): Buczinski, Stephen C.
• Application Number:
  US09/401,059
• Time in Patent Office: 797 Days
• Field of Search: 356/5-1-, 356/141.1
• US Class Current: 356/5.01
• CPC Class Codes: G01C 3/08 Use of electric radiation detectors
• CPC Class Codes: G01S 17/10 using transmission of interrupted, pulse-modulated waves determination of distance by phase measurements G01S17/32
• CPC Class Codes: G01S 17/14 wherein a voltage or current pulse is initiated and terminated in accordance with the pulse transmission and echo reception respectively, e.g. using counters
• CPC Class Codes: G01S 17/46 Indirect determination of position data
• CPC Class Codes: G01S 17/894 3D imaging with simultaneous measurement of time-of-flight at a 2D array of receiver pixels, e.g. time-of-flight cameras or flash lidar
• CPC Class Codes: G01S 7/4802 using analysis of echo signal for target characterisation; Target signature; Target cross-section
• CPC Class Codes: G01S 7/486 Receivers
• CPC Class Codes: G01S 7/497 Means for monitoring or calibrating
• CPC Class Codes: G06V 30/228 of three-dimensional handwriting, e.g. writing in the air
• CPC Class Codes: H04N 25/53 Control of the integration time
• CPC Class Codes: H04N 3/155 Control of the image-sensor operation, e.g. image processing within the image-sensor

Abstract

A three-dimensional imaging system includes a two-dimensional array of pixel light sensing detectors and dedicated electronics and associated processing circuitry fabricated on a common IC using CMOS fabrication techniques. In one embodiment, each detector has an associated high speed counter that accumulates clock pulses in number directly proportional to time of flight (TOF) for a system-emitted pulse to reflect from an object point and be detected by a pixel detector focused upon that point. The TOF data provides a direct digital measure of distance from the particular pixel to a point on the object reflecting the emitted light pulse. In a second embodiment, the counters and high speed clock circuits are eliminated, and instead each pixel detector is provided with a charge accumulator and an electronic shutter. The shutters are opened when a light pulse is emitted and closed thereafter such that each pixel detector accumulates charge as a function of return photon energy falling upon the associated pixel detector. The amount of accumulated charge provides a direct measure of round-trip TOF. In either embodiment, the collection of TOF data permits reconstruction of the three-dimensional topography of the light-reflecting surface of the object being imaged. The CMOS nature of the array permits random-order readout of TOF data if desired. Using a light source of a known wavelength and filtering out incoming light of other wavelengths permits use of the system with or without ambient light.

Claims

1. A method to determine distance Z within resolution Δ

Z between at least one pixel detector and a target less than about 100 m distant, the method comprising the following steps;

(a) disposing said at least one pixel detector so as to detect optical energy reflected from said target, each said pixel detector having a pixel integration delay time exceeding 2·

Δ

Z/C, where C is speed of light, and for each said pixel detector disposing an associated pixel time interval counter;

(b) illuminating said target at time t0 with at least a first pulse of optical energy from an energy source;

(c) starting each said pixel time interval counter at said time t0 to count a clock signal operating at frequency at least about C/(2·

Δ

Z);

(d) halting each said pixel time interval counter when an associated said pixel detector detects optical energy reflected from said target resulting from said first pulse;

wherein detection time for said pixel detector includes actual time of flight (TOF) from said energy source to said target to said pixel detector, as well as pixel integration delay time for said pixel detector; and

(e) removing said pixel integration delay time on a per pixel detector basis to obtain said actual TOF, whereby said distance Z is obtained.

2. The method of claim 1, wherein step (a) includes disposing an array of pixel detectors, each pf said pixel detectors having a pixel integration delay time exceeding 2·

Δ

Z/C, where C is speed of light;

said array including for each of said pixel detectors an associated pixel time interval counter.

3. The method of claim 1, wherein step (e) results in a direct digital measurement of time when said pixel detector detects energy.

4. The method of claim 1, wherein said clock signal exceeds 100 MHz.

5. The method of claim 1, wherein said pixel time interval counter is a pseudo random sequence digital counter.

6. The method of claim 1, wherein said pixel time interval counter is a true single phase clocked pseudo random sequence digital counter.

7. The method of claim 1, wherein said pixel time interval counter includes sub-clock period capacitor charging.

8. The method of claim 2, wherein information detected by said pixel detectors in said array may be randomly readout from said array.

9. The method of claim 1, wherein step (b) includes outputting optical energy pulses of varying amplitude, and wherein step (e) uses reflected energy from said pulses of varying amplitude to determine said pixel integration delay time.

10. The method of claim 1, further including:

outputting multiple optical energy pulses; and

averaging detection time for each said pulse to enhance signal/noise ratio in said determining said distance Z.

11. The method of claim 1, wherein at least one of step (a), step (c), step (d), and step (e) are carried out on a common integrated circuit substrate.

12. A CMOS-implementable integrated circuit (IC) time of flight (TOF) measurement system used with a generator that outputs light pulses commencing at time t0 to determine distance Z within resolution Δ

Z between said IC and a target less than about 100 m distant, the IC including;

an array comprising a plurality of pixel detectors to detect generated light pulse energy returned from said target, each of said pixel detectors having a pixel integration delay time exceeding 2·

Δ

Z/C, where C is speed of light, said array further comprising for each of said pixel detectors an associated pixel time interval counter;

a system clock providing countable clock pulses to each said pixel time interval counter, said clock having a frequency at least about C/(2·

Δ

Z);

logic to enable each said pixel time interval counter to count said clock pulses commencing with start of a generated light pulse at said time t0, and to disable each said pixel time interval counter from further counting clock pulses when an associated pixel detector detects energy from the generated said light pulse;

wherein detection time for each said pixel detector includes actual time of flight (TOF) from said generator to said target to said pixel detector, as well as pixel integration delay time for said pixel detector; and

wherein said pixel integration delay time may be removed on a per pixel detector basis to obtain said actual TOF such that a direct digital measurement proportional to said distance Z is obtained.

13. The IC of claim 12, wherein:

said system clock has at least one characteristic selected from a group consisting of (a) a clock rate exceeding about 100 MHz, (b) said clock pulses are single-phase clock pulses, (c) said clock compensates for clock pulse phase at each of said pixel detectors, and (d) said clock includes means for maintaining relative clock edge integrity.

14. The IC of claim 12, wherein each said pixel time interval counter is a pseudo random sequence counter.

15. The IC of claim 12, further including means for providing sub-clock period capacitor charging to measure output of each said pixel time interval counter.

16. The IC of claim 12, further including means to average detection time of multiple said pulses to enhance signal/noise ratio in determining said distance Z.

17. The IC of claim 12, further including means for collecting data over multiple light pulses, said means for collecting data including at least one of (a) a look-up calibration table for said IC to compensate for said pixel integration delay, and (b) means for examining characteristics associated with various of said pixel detectors to compensate for said pixel integration delay.

18. The IC of claim 12, further including a processor to process outputs from each said pixel time interval counter to provide Z distance data relating to said target.

19. The IC of claim 18, wherein said processor can determine at least one of (a) said Z distance from said IC to said target, (b) velocity of said target, (c) relative shape of said target, and (d) relative identification of said target.

20. A method to determine distance Z between a detector array and a target less than about 100 m distant, the method comprising the following steps:

(a) disposing said array so as to detect optical energy reflected from said target, the array comprising a plurality of pixel detectors, and for each pixel detector providing an associated charge collector, and for each pixel detector an associated shutter disposed intermediate an associated said pixel detector and associated said charge collector to control ability of said charge collector to collect charge output by an associated said pixel detector;

(b) at time t0, illuminating said target with a first pulse of optical energy having pulse width PW and opening each said shutter to permit said collector to collect any output resulting from detected optical energy reflected from said target resulting from said pulse;

(c) beginning at said time t0 to accumulate in each said collector any charge output by an associated said pixel detector;

(d) at a time Tep, corresponding to approximately PW=Tep−

t0, closing each said shutter to disable each said associated collector from accumulating further charge;

(e) after said time Tep, evaluating charge associated in each said collector to obtain a measure of actual time of flight (TOF) for each said pixel detector to said target; and

(f) at a subsequent time outputting a second pulse of known PW and opening and retaining open each said shutter until cessation of detected energy from said second pulse;

wherein said distance Z to said target may be determined.

21. The method of claim 20, further including accumulating charge in each collector using at least two pulses of known PW with said shutter enabled, and accumulating charge in said collector using at least two pulses of said known PW with said shutter disabled to obtain calibration data representing light reflectivity of said target;

wherein measurement accuracy is enhanced.

22. The method of claim 20, further including accumulating charge in at least one of said pixel detectors to obtain calibration data representing reflectivity of said target.

23. A CMOS-implementable integrated circuit (IC) time of flight (TOF) measurement system used with a generator outputting at least a first light pulse of pulse width PW to determine distance Z between said IC and a target less than about 100 m distant, the IC comprising:

an array of pixel detectors to detect generated light pulse energy returned from said target, each pixel detector having a pixel integration delay time exceeding 2·

Δ

Z/C where C is speed of light;

for each of said pixel detectors, an associated charge collector, and for each pixel detector an associated shutter disposed intermediate an associated said pixel detector and associated said charge collector to control ability of said charge collector to collect charge output by an associated said pixel detector;

logic that opens each said shutter at a time t0 representing start of a generated said first light pulse to permit said collector to collect any output resulting from detected optical energy reflected from said target resulting from said first pulse, said logic closing each said shutter at a time Tep corresponding to approximately PW=Tep−

t0 width of each said first pulse to disable each said associated collector from accumulating further charge;

means for evaluating accumulated charge after said time Tep accumulated in each said collector to obtain a measure of actual time of flight (TOF) for each said pixel detector to said target;

said logic further opening and retaining open each said shutter, at a subsequent time corresponding to output duration of a second pulse of known pulse width, until cessation of detected energy from said second pulse;

wherein said distance Z to said target may be determined.

24. The IC of claim 23, wherein said logic permits opening each said shutter to obtain calibration data representing light reflectivity of said target.

25. The IC of claim 23, further including a microprocessor to determine from evaluated said accumulated charge at least one of (a) distance Z from said IC to said target, (b) velocity of said target, (c) relative shape of said target, and (d) relative identification of said target.