Method and system for automatic gain control of sensors in time-of-flight systems

• US 7,511,801 B1
• Filed: 05/27/2008
• Issued: 03/31/2009
• Est. Priority Date: 02/08/2005
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Canesta Incorporated (Microsoft Corporation)
• Inventors: Gokturk, Salih Burak, Rafii, Abbas
• Primary Examiner(s): Tarcza; Thomas H
• Assistant Examiner(s): Ratcliffe; Luke D
• Application Number:
  US12/154,774
• Time in Patent Office: 308 Days
• Field of Search: 356 301- 315, 356 401- 41, 356 501- 515, 356 6- 22, 356/28, 356/28.5
• US Class Current: 356/5.01
• CPC Class Codes: G01C 3/08 Use of electric radiation detectors

Abstract

Performance of pixel detectors in a TOF imaging system is dynamically adjusted to improve dynamic range to maximize the number of pixel detectors that output valid data. The invention traverses the system-acquired z depth, the brightness, and the active brightness images, and assigns each pixel a quantized value. Quantization values encompass pixels receiving too little light, normal light, to too much light. Pixels are grouped into quantized category groups, whose populations are represented by a histogram. If the number of pixels in the normal category exceeds a threshold, no immediate corrective action is taken. If the number of pixel receiving too little (or too much) light exceeds those receiving too much (or too little) light, the invention commands at least one system parameter change to increase (or decrease) light reaching the pixels. Controllable TOF system parameters can include exposure time, common mode resets, video gain, among others.

Claims

1. For use with a time-of-flight (TOF) system that illuminates a target, a distance z from said TOF system, with emitted optical energy having a modulated periodic waveform that includes a high frequency component, and that detects optical energy including ambient light and at least a fraction of said emitted optical energy with an array of pixel photodetectors to determine data proportional to said distance z, said TOF system operable independently of reflectivity of said target, a method to automatically control detection gain of said pixel photodetectors, the method comprising the following steps:

(a) identifying which of said pixel photodetectors are receiving excessive levels of said optical energy to output reliable detection data, which of said pixel photodetectors are receiving insufficient levels of said optical energy to output reliable detection data, and which of said pixel photodetectors are receiving adequate levels of said optical energy to output reliable detection data;

(b) designating one of at least three groups into which pixel photodetectors identified in step (a) are placed, comprising (i) said pixel photodetectors receiving excessive levels of said optical energy, (ii) said pixel photodetectors receiving insufficient levels of said optical energy, and (iii) said pixel photodetectors receiving adequate levels of said optical energy; and

(c) varying at least one system parameter associated with said TOF system to alter levels of said optical energy presented to said pixel detectors so as to increase number of pixel photodetectors in a group of pixel photodetectors receiving adequate levels of said optical energy to output reliable detection data.

2. The method of claim 1, wherein said at least one system parameter is selected from a group consisting of pixel photodetector integration time, number of detection common mode resets, and video gain.

3. The method of claim 1, wherein at least one of step (a) and step (b) is carried out in software executable by a processor.

4. The method of claim 1, wherein at step (a), identification of said pixel photodetectors receiving excessive levels of said optical energy is determined from a passive brightness image acquired by said TOF system.

5. The method of claim 1, wherein at step (a), identification of said pixel photodetectors receiving insufficient levels of said optical energy is determined from an active brightness image acquired by said TOF system.

6. The method of claim 1, wherein at least two of steps (a), (b), and (c) are implemented in software storable in memory associated with said TOF system and executable by a microprocessor associated with said TOF system.

7. For use with a time-of-flight (TOF) system that illuminates a target, a distance z from said TOF system, with emitted optical energy having a modulated periodic waveform that includes a high frequency component, and that detects optical ambient light and at least a fraction of said emitted optical energy with an array of pixel photodetectors operable to determine data proportional to said distance z, said TOF system operable independently of reflectivity of said target, a control system to automatically control detection gain of said pixel photodetectors, the control system comprising:

means for identifying which of said pixel photodetectors are receiving excessive levels of said optical energy to output reliable detection data, which of said pixel photodetectors are receiving insufficient levels of said optical energy to output reliable detection data, and which of said pixel photodetectors are receiving adequate levels of said optical energy to output reliable detection data;

means for designating one of at least three groups into which pixel photodetectors identified by said means for identifying are placed, designation choices comprising (i) said pixel photodetectors receiving excessive levels of said optical energy, (ii) said pixel photodetectors receiving insufficient levels of said optical energy, and (iii) said pixel photodetectors receiving adequate levels of said optical energy; and

means for varying at least one system parameter associated with said TOF system;

wherein said means for varying varies at least one system parameter associated with said TOF system to alter levels of said optical energy presented to said pixel detectors so as to increase number of pixel photodetectors in a group of pixel photodetectors receiving adequate levels of said optical energy to output reliable detection data.

8. The control system claim 7, wherein said at least one system parameter is selected from a group consisting of pixel photodetector integration time, number of detection common mode resets, and video gain.

9. The control system of claim 7, wherein at least one of said means for identifying and said means for designating is implemented in software executable by a processor associated with said TOF system.

10. The control system of claim 7, wherein said means for identifying uses a passive brightness image acquired by said TOF system to determine which if any of said pixel photodetectors are receiving excessive levels of said optical energy.

11. The control system of claim 7, wherein said means for identifying uses an active brightness image acquired by said TOF system to determine which if any of said pixel photodetectors are receiving insufficient levels of said optical energy.

12. For use with a time-of-flight (TOF) system that illuminates a target, a distance z from said TOF system, with optical energy having a modulated periodic waveform that includes a high frequency component, and that detects optical energy including ambient light and at least a fraction of said illumination with an array of pixel photodetectors to determine data proportional to said distance z, said TOF system operable independently of ambient light, a method to automatically control detection gain of said pixel photodetectors, the method comprising the following steps:

(a) identifying from a passive brightness image acquired by said TOF system which of said pixel photodetectors are receiving excessive levels of said optical energy to output reliable detection data, and identifying from an active brightness image which of said pixel photodetectors are receiving insufficient levels of said optical energy to output reliable detection data and forming a histogram of such identification;

(b) using identification information from step (a) to form a histogram representing number of said pixel photodetectors receiving excessive levels of said optical energy, number of said pixel photodetectors receiving insufficient levels of said optical energy, and number of said pixel photodetectors receiving neither excessive nor insufficient levels of said optical energy;

(c) estimating variation in at least one system parameter of said TOF system to produce a change in light Δ

L that increases said number of said pixel photodetectors receiving neither excessive nor insufficient levels of said optical energy; and

(d) varying at least one system parameter of said TOF system to increase said number of said pixel photodetectors receiving neither excessive nor insufficient levels of said optical energy.

13. The method of claim 12, wherein said at least one system parameter is selected from a group consisting of pixel photodetector integration time, number of detection common mode resets, and video gain.

14. The method of claim 12, wherein at least two of steps (a), (b), and (c) are implemented in software executable by a processor.

15. For use with a time-of-flight (TOF) system that illuminates a target, a distance z from said TOF system, with emitted optical energy having a modulated periodic waveform that includes a high frequency component, and that detects optical energy including ambient light and at least a fraction of said emitted optical energy with an array of pixel photodetectors to determine data proportional to said distance z, said TOF system operable independently of ambient light, a control system to automatically control detection gain of said pixel photodetectors, said control system comprising:

means for identifying from a passive brightness image acquired by said TOF system which of said pixel photodetectors are receiving excessive levels of said energy to output reliable detection data, and for identifying from an active brightness image which of said pixel photodetectors are receiving insufficient levels of said optical energy to output reliable detection data and forming a histogram of such identification;

means for forming a histogram, using identification information from said means for identifying, representing number of said pixel photodetectors receiving excessive levels of said optical energy, number of said pixel photodetectors receiving insufficient levels of said optical energy, and number of said pixel photodetectors receiving neither excessive nor insufficient levels of said optical energy;

means for estimating variation in at least one system parameter of said TOF system to produce a change in light Δ

L that increases said number of said pixel photodetectors receiving neither excessive nor insufficient levels of said optical energy; and

means for varying at least one system parameter of said TOF system to increase said number of said pixel photodetectors receiving neither excessive nor insufficient levels of said optical energy.

16. The control system of claim 15, wherein said at least one system parameter is selected from a group consisting of pixel photodetector integration time, number of detection common mode resets, and video gain.

17. The control system of claim 15, wherein at least two of said means for identifying, said means for forming, and said means for estimating are implemented in software executable by a processor.

18. For use with a time-of-flight (TOF) system that illuminates a target, a distance z from said TOF system, with emitted optical energy having a modulated periodic waveform that includes a high frequency component, and that detects optical energy including ambient light and at least a fraction of said emitted optical energy with an array of pixel photodetectors to determine data proportional to said distance z, said TOF system operable independently of ambient light, a method to automatically control detection gain of said pixel photodetectors, the method comprising the following steps:

(a) identifying from a passive brightness image acquired by said TOF system which of said pixel photodetectors are receiving excessive levels of said optical energy to output reliable detection data, and identifying from an active brightness image which of said pixel photodetectors are receiving insufficient levels of said optical energy to output reliable detection data and forming a histogram of such identification;

(b) using identification information from step (a) to traverse said active brightness image and said passive brightness image to determine therefrom number of said pixel photodetectors receiving insufficient levels of said optical energy, and number of said pixel photodetectors receiving neither excessive nor insufficient levels of said optical energy;

(c) estimating variation in at least one system parameter of said TOF system to produce a change in light Δ

L that increases said number of said pixel photodetectors receiving neither excessive nor insufficient levels of said optical energy; and

(d) varying at least one system parameter of said TOF system to increase said number of said pixel photodetectors receiving neither excessive nor insufficient levels of said optical energy.

19. The method of claim 18, wherein said at least one system parameter is selected from a group consisting of pixel photodetector integration time, number of detection common mode resets, and video gain.

20. For use with a time-of-flight (TOF) system that illuminates a target, a distance z from said TOF system, with optical energy having a modulated periodic waveform that includes a high frequency component, and that detects optical energy including ambient light and at least a fraction of said illumination with an array of pixel photodetectors to determine data proportional to said distance z, a control system to automatically control detection gain of said pixel photodetectors, said TOF system operable independently of reflectivity of said target, the control system comprising:

means for identifying from a passive brightness image acquired by said TOF system which of said pixel photodetectors are receiving excessive levels of said optical energy to output reliable detection data, and identifying from an active brightness image which of said pixel photodetectors are receiving insufficient levels of said optical energy to output reliable detection data and forming a histogram of such identification;

means for traversing said active brightness image and said passive brightness image and for determining therefrom number of said pixel photodetectors receiving insufficient levels of said optical energy, and number of said pixel photodetectors receiving neither excessive nor insufficient levels of said optical energy;

means for estimating variation in at least one system parameter of said TOF system to produce a change in light Δ

L that increases said number of said pixel photodetectors receiving neither excessive nor insufficient levels of said optical energy; and

means for varying at least one system parameter of said TOF system to increase said number of said pixel photodetectors receiving neither excessive nor insufficient levels of said optical energy.