Method and system for automatic gain control of sensors in time-of-flight systems

• US 7,379,163 B2
• Filed: 02/06/2006
• Issued: 05/27/2008
• Est. Priority Date: 02/08/2005
• Status: Expired due to Fees

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Canesta Incorporated (Microsoft Corporation)
• Inventors: Gokturk, Salih Burak, Rafii, Abbas
• Primary Examiner(s): Tarcza; Thomas H.
• Assistant Examiner(s): RATCLIFFE, LUKE D
• Application Number:
  US11/349,310
• Publication Number:
  US 20060176467A1
• Time in Patent Office: 841 Days
• Field of Search: 356/4.07
• US Class Current: 356/4.07
• CPC Class Codes: G01C 3/08 Use of electric radiation detectors

Abstract

Performance of pixel detectors in a TOF imaging system is dynamically adjusted to improve dynamic range to maximize the number of pixel detectors that output valid data. The invention traverses the system-acquired z depth, the brightness, and the active brightness images, and assigns each pixel a quantized value. Quantization values encompass pixels receiving too little light, normal light, to too much light. Pixels are grouped into quantized category groups, whose populations are represented by a histogram. If the number of pixels in the normal category exceeds a threshold, no immediate corrective action is taken. If the number of pixel receiving too little (or too much) light exceeds those receiving too much (or too little) light, the invention commands at least one system parameter change to increase (or decrease) light reaching the pixels. Controllable TOF system parameters can include exposure time, common mode resets, video gain, among others.

Claims

1. For use with a time-of-flight (TOF) system that illuminates a target, a distance z from said TOF system, with emitted optical energy having a modulated periodic waveform that includes a high frequency component, and that detects a fraction of said emitted optical energy with an array of pixel photodetectors operable in at least two phases to determine data proportional to said distance z, said TOF system operable to determine distance z independently of reflectivity of said target, a method to automatically control detection gain of said pixel photodetectors, the method comprising the following steps:

(a) identifying which of said pixel photodetectors are receiving excessive levels of emitted optical energy to output reliable detection data, which of said pixel photodetectors are receiving insufficient levels of emitted optical energy to output reliable detection data, and which of said pixel photodetectors are receiving adequate levels of emitted optical energy to output reliable detection data;

(b) designating one of at least three groups into which pixel photodetectors identified in step (a) are placed, comprising (i) said pixel photodetectors receiving excessive levels of emitted optical energy, (ii) said pixel photodetectors receiving insufficient levels of emitted optical energy, and (iii) said pixel photodetectors receiving adequate levels of emitted optical energy; and

(c) if a highest number of said pixel photodetectors are members of a group comprising (i) said pixel photodetectors receiving excessive levels of emitted optical energy, varying at least one system parameter associated with said TOF system to reduce levels of emitted optical energy presented to said pixel photodetectors; and

if a highest number of said pixel photodetectors are members of a group comprising (ii) said pixel photodetectors receiving insufficient levels of emitted optical energy, varying at least one system parameter associated with said TOF system to increase levels of emitted optical energy presented to said pixel photodetectors.

2. The method of claim 1, wherein said at least one system parameter is selected from a group consisting of pixel photodetector integration time, number of detection common mode resets, and video gain.

3. The method of claim 1, wherein at least one of step (a) and step (b) is carried out in software executable by a processor.

4. The method of claim 1, wherein at step (a), identification of said pixel photodetectors receiving excessive levels of emitted optical energy is determined from a passive brightness image acquired by said TOF system.

5. The method of claim 1, wherein at step (a), identification of said pixel photodetectors receiving insufficient levels of emitted optical energy is determined from an active brightness image acquired by said TOF system.

6. The method of claim 1, wherein at least two of steps (a), (b), and (c) are implemented in software storable in memory associated with said TOF system and executable by a microprocessor associated with said TOF system.

7. For use with a time-of-flight (TOF) system that illuminates a target, a distance z from said TOF system, with emitted optical energy having a modulated periodic waveform that includes a high frequency component, and that detects a fraction of said emitted optical energy with an array of pixel photodetectors operable in at least two phases to determine data proportional to said distance z, said TOF system operable to determine distance z independently of reflectivity of said target, a control system to automatically control detection gain of said pixel photodetectors, the control system comprising:

means for identifying which of said pixel photodetectors are receiving excessive levels of emitted optical energy to output reliable detection data, which of said pixel photodetectors are receiving insufficient levels of emitted optical energy to output reliable detection data, and which of said pixel photodetectors are receiving adequate levels of emitted optical energy to output reliable detection data;

means for designating one of at least three groups into which pixel photodetectors identified by said means for identifying are placed, designation choices comprising (i) said pixel photodetectors receiving excessive levels of emitted optical energy, (ii) said pixel photodetectors receiving insufficient levels of emitted optical energy, and (iii) said pixel photodetectors receiving adequate levels of emitted optical energy; and

means for varying at least one system parameter associated with said TOF system;

wherein said means for varying varies at least one system parameter associated with said TOF system to reduce levels of emitted optical energy presented to said pixel photodetectors if said means for designating determines that a highest number of said pixel photodetectors are members of a group comprising said pixel photodetectors receiving excessive levels of emitted optical energy; and

wherein said means for varying varies at least one system parameter associated with said TOF system to increase levels of emitted optical energy presented to said pixel photodetectors if a highest number of said pixel photodetectors are members of a group comprising said pixel photodetectors receiving insufficient levels of emitted optical energy.

8. The control system claim 7, wherein said at least one system parameter is selected from a group consisting of pixel photodetector integration time, number of detection common mode resets, and video gain.

9. The control system of claim 7, wherein at least one of said means for identifying and said means for designating is implemented in software executable by a processor associated with said TOF system.

10. The control system of claim 7, wherein said means for identifying uses a passive brightness image acquired by said TOF system to determine which if any of said pixel photodetectors are receiving excessive levels of emitted optical energy.

11. The control system of claim 7, wherein said means for identifying uses an active brightness image acquired by said TOF system to determine which if any of said pixel photodetectors are receiving insufficient levels of emitted optical energy.

12. For use with a time-of-flight (TOF) system that illuminates a target, a distance z from said TOF system, with optical energy having a modulated periodic waveform that includes a high frequency component, and that detects a fraction of said illumination with an array of pixel photodetectors operable in at least two phases to determine data proportional to said distance z, said TOF system operable to determine distance z independently of ambient light, a method to automatically control detection gain of said pixel photodetectors, the method comprising the following steps:

(a) identifying from a passive brightness image acquired by said TOF system which of said pixel photodetectors are receiving excessive levels of emitted optical energy to output reliable detection data, and identifying from an active brightness image which of said pixel photodetectors are receiving insufficient levels of emitted optical energy to output reliable detection data and forming a histogram of such identification;

(b) using identification information from step (a) to form a histogram representing number of said pixel photodetectors receiving excessive levels of emitted optical energy, number of said pixel photodetectors receiving insufficient levels of emitted optical energy, and number of said pixel photodetectors receiving neither excessive nor insufficient levels of emitted optical energy;

(c) estimating variation in at least one system parameter of said TOF system to produce a change in light Δ

L that increases said number of said pixel photodetectors receiving neither excessive nor insufficient levels of emitted optical energy;

(d) varying at least one system parameter of said TOF system to increase said number of said pixel photodetectors receiving neither excessive nor insufficient levels of emitted optical energy.

13. The method of claim 12, wherein said at least one system parameter is selected from a group consisting of pixel photodetector integration time, number of detection common mode resets, and video gain.

14. The method of claim 12, wherein at least two of steps (a), (b), and (c) are implemented in software executable by a processor.

15. For use with a time-of-flight (TOF) system that illuminates a target, a distance z from said TOF system, with emitted optical energy having a modulated periodic waveform that includes a high frequency component, and that detects a fraction of said emitted optical energy with an array of pixel photodetectors operable in at least two phases to determine data proportional to said distance z, said TOF system operable to determine distance z independently of ambient light, a control system to automatically control detection gain of said pixel photodetectors, said control system comprising:

means for identifying from a passive brightness image acquired by said TOF system which of said pixel photodetectors are receiving excessive levels of emitted optical energy to output reliable detection data, and for identifying from an active brightness image which of said pixel photodetectors are receiving insufficient levels of emitted optical energy to output reliable detection data and forming a histogram of such identification;

means for forming a histogram, using identification information from said means for identifying, representing number of said pixel photodetectors receiving excessive levels of emitted optical energy, number of said pixel photodetectors receiving insufficient levels of optical energy, and number of said pixel photodetectors receiving neither excessive nor insufficient levels of emitted optical energy;

means for estimating variation in at least one system parameter of said TOF system to produce a change in light Δ

L that increases said number of said pixel photodetectors receiving neither excessive nor insufficient levels of emitted optical energy; and

means for varying at least one system parameter of said TOF system to increase said number of said pixel photodetectors receiving neither excessive nor insufficient levels of emitted optical energy.

16. The control system of claim 15, wherein said at least one system parameter is selected from a group consisting of pixel photodetector integration time, number of detection common mode resets, and video gain.

17. The control system of claim 15, wherein at least two of said means for identifying, said means for forming, and said means for estimating are implemented in software executable by a processor.

18. For use with a time-of-flight (TOF) system that illuminates a target, a distance z from said TOF system, with emitted optical energy having a modulated periodic waveform that includes a high frequency component, and that detects a fraction of said emitted optical energy with an array of pixel photodetectors operable in at least two phases to determine data proportional to said distance z, said TOF system operable independently of ambient light, a method to automatically control detection gain of said pixel photodetectors, the method comprising the following steps:

(a) identifying from a passive brightness image acquired by said TOF system which of said pixel photodetectors are receiving excessive levels of emitted optical energy to output reliable detection data, and identifying from an active brightness image which of said pixel photodetectors are receiving insufficient levels of emitted optical energy to output reliable detection data and forming a histogram of such identification;

(b) using identification information from step (a) to traverse said active brightness image and said passive brightness image to determine therefrom number of said pixel photodetectors receiving insufficient levels of emitted optical energy, and number of said pixel photodetectors receiving neither excessive nor insufficient levels of emitted optical energy;

(c) estimating variation in at least one system parameter of said TOF system to produce a change in light Δ

L that increases said number of said pixel photodetectors receiving neither excessive nor insufficient levels of emitted optical energy; and

(d) varying at least one system parameter of said TOF system to increase said number of said pixel photodetectors receiving neither excessive nor insufficient levels of emitted optical energy.

19. The method of claim 18, wherein said at least one system parameter is selected from a group consisting of pixel photodetector integration time, number of detection common mode resets, and video gain.

20. For use with a time-of-flight (TOF) system that illuminates a target, a distance z from said TOF system, with optical energy having a modulated periodic waveform that includes a high frequency component, and that detects a fraction of said illumination with an array of pixel photodetectors operable in at least two phases to determine data proportional to said distance z, said TOF system operable independently of reflectivity of said target, a control system to automatically control detection gain of said pixel photodetectors, the control system comprising:

means for identifying from a passive brightness image acquired by said TOF system which of said pixel photodetectors are receiving excessive levels of emitted optical energy to output reliable detection data, and identifying from an active brightness image which of said pixel photodetectors are receiving insufficient levels of emitted optical energy to output reliable detection data and forming a histogram of such identification;

means for traversing said active brightness image and said passive brightness image and for determining therefrom number of said pixel photodetectors receiving insufficient levels of emitted optical energy, and number of said pixel photodetectors receiving neither excessive nor insufficient levels of emitted optical energy;

means for estimating variation in at least one system parameter of said TOF system to produce a change in light Δ

L that increases said number of said pixel photodetectors receiving neither excessive nor insufficient levels of emitted optical energy; and

means for varying at least one system parameter of said TOF system to increase said number of said pixel photodetectors receiving neither excessive nor insufficient levels of emitted optical energy.