Methods and system to quantify depth data accuracy in three-dimensional sensors using single frame capture

• US 20060197937A1
• Filed: 02/06/2006
• Published: 09/07/2006
• Est. Priority Date: 02/08/2005
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Canesta Incorporated (Microsoft Corporation)
• Inventors: Torunoglu, Iihami, Gokturk, Salih Burak, Bamji, Cyrus
• Granted Patent: US 7,408,627 B2
• Time in Patent Office: Days
• US Class Current: 356/5.01
• CPC Class Codes: G01C 3/08 Use of electric radiation detectors
• CPC Class Codes: G01S 17/894 3D imaging with simultaneous measurement of time-of-flight at a 2D array of receiver pixels, e.g. time-of-flight cameras or flash lidar
• CPC Class Codes: G01S 7/497 Means for monitoring or calibrating

Abstract

A method and system dynamically calculates confidence levels associated with accuracy of Z depth information obtained by a phase-shift time-of-flight (TOF) system that acquires consecutive images during an image frame. Knowledge of photodetector response to maximum and minimum detectable signals in active brightness and total brightness conditions is known a priori and stored. During system operation brightness threshold filtering and comparing with the a priori data permits identifying those photodetectors whose current output signals are of questionable confidence. A confidence map is dynamically generated and used to advise a user of the system that low confidence data is currently being generated. Parameter(s) other than brightness may also or instead be used.

Claims

1. For use in a phase-shift time-of-flight (TOF) system that emits signals of optical energy and determines distance Z to a target object by detecting and examining relative phase shift in optical energy signals reflected from the target object and detected by an array of pixel photodetectors, a method to assign a measure of confidence to output signals detected by said photodetectors, the method including the following steps:

(a) acquiring during a time frame from said photodetectors active brightness data and total brightness data;

(b) storing a priori-determined at least one of upper and lower detectable levels of optical energy for at least some of said photodetectors;

(c) comparing levels stored at step (b) with presently generated output signals from said at least some photodetectors to identify which, if any, of said photodetectors is presently operating other than in an acceptable dynamic range; and

(d) characterizing Z depth information obtained by output from said at least some detectors as being at least one of questionable confidence and acceptable confidence.

2. The method of claim 1, further including:

(e) indicating to a user of said TOF system when questionable confidence data is being acquired.

3. The method of claim 1, wherein an active brightness signal for said pixel photodetectors, when operating within an acceptable dynamic range, is given by:

Signal=Active_Brightness=√

{square root over ((Δ

0°

−

Δ

180°

)²+(Δ

90°

−

Δ

270°

)²)}

where said TOF system acquires data at least at phases of 0°

, 189°

, 90°

, and 270°

.

4. The method of claim 1, wherein a signal/noise ratio of zero and a confidence level of zero is assigned to any of said photodetectors determined to be operating other than within an acceptable dynamic range.

5. The method of claim 1, wherein step (c) is carried out using at least one brightness-based threshold filter.

6. The method of claim 1, wherein step (c) is carried out in at least one of hardware and software.

7. The method of claim 1, wherein step (e) includes displaying a user-viewable image, which image indicates low confidence data are used in creating said image.

8. The method of claim 1, wherein step (e) includes generating a user-detectable acoustic signal indicating low confidence data are presently being used.

9. The method of claim 1, wherein step (b) includes storing levels associated with at least one system environmental parameter selected from a group consisting of system temperature, system operating potential, system altitude, and system pressure.

10. The method of claim 1, wherein at least one of step (c) and step (d) are carried out by a microprocessor associated with said TOF system executing software stored in memory associated with said TOF system.

11. For use in a phase-shift time-of-flight (TOF) system that emits signals of optical energy and determines distance Z to a target object by detecting and examining relative phase shift in optical energy signals reflected from the target object and detected by an array of pixel photodetectors, said TOF system acquiring active brightness data and total brightness data, a confidence system to assign a measure of confidence to output signals detected by said photodetectors, the confidence system comprising:

memory to store a priori-determined at least one of upper and lower detectable levels of optical energy for at least some of said photodetectors;

means for comparing at least some of said stored said upper and levels detectable levels with presently generated output signals from said at least some photodetectors to identify which, if any, of said photodetectors is presently operating other than in an acceptable dynamic range; and

means for characterizing Z depth information obtained by output from said at least some detectors as being at least one of questionable confidence and acceptable confidence.

12. The confidence system of claim 11, further including means for indicating to a user of said TOF system when questionable confidence data is being acquired.

13. The confidence system of claim 11, wherein an active brightness signal for said pixel photodetectors, when operating within an acceptable dynamic range, is given by:

Signal=Active_Brightness=√

{square root over ((Δ

0°

−

Δ

180°

)²+(Δ

90°

−

Δ

270°

)²)}

where said TOF system acquires data at least at phases of 0°

, 189°

, 90°

, and 270°

.

14. The confidence system of claim 11, wherein a signal/noise ratio of zero and a confidence level of zero is assigned to any of said photodetectors determined to be operating other than within an acceptable dynamic range.

15. The confidence system of claim 11, further including software, storable in memory in said TOF system, that when executed implements at least one of said means for comparing and said means for characterizing.

16. The confidence system of claim 15, wherein said software is executable by a processor in said TOF system.

17. The confidence system of claim 12, wherein said means for indicating includes at least one of displaying a user-viewable image, which image indicates low confidence data are used in creating said image, and generating a user-detectable acoustic signal indicating low confidence data are presently being used.

18. The confidence system of claim 12, wherein said memory stores levels associated with at least one system environmental parameter selected from a group consisting of system temperature, system operating potential, system altitude, and system pressure.

19. A phase-shift time-of-flight (TOF) system that emits signals of optical energy and determines distance Z to a target object by detecting and examining relative phase shift in optical energy signals reflected from the target object and detected by an array of pixel photodetectors wherein said TOF system acquires active brightness data and total brightness data, said TOF system further including:

a confidence system to assign a measure of confidence to output signals detected by said photodetectors, the confidence system comprising;

memory to store a priori-determined at least one of upper and lower detectable levels of optical energy for at least some of said photodetectors;

means for comparing at least some of said stored said upper and levels detectable levels with presently generated output signals from said at least some photodetectors to identify which, if any, of said photodetectors is presently operating other than in an acceptable dynamic range; and

means for characterizing Z depth information obtained by output from said at least some detectors as being at least one of questionable confidence and acceptable confidence.

20. The TOF system of claim 19, further including means for indicating to a user of said TOF system when questionable confidence data is being acquired.