MULTIPATH SIGNAL REMOVAL IN TIME-OF-FLIGHT CAMERA APPARATUS

• US 20170322310A1
• Filed: 05/09/2016
• Published: 11/09/2017
• Est. Priority Date: 05/09/2016
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Mirko Schmidt
• Inventors: Godbaz, John Peter, Schmidt, Mirko, Bamji, Cyrus S.
• Granted Patent: US 10,302,768 B2
• Time in Patent Office: Days
• CPC Class Codes: G01S 17/08 for measuring distance only indirect measurement G01S17/46; active triangulation systems G01S17/48
• CPC Class Codes: G01S 17/50 Systems of measurement based on relative movement of target
• CPC Class Codes: G01S 17/894 3D imaging with simultaneous measurement of time-of-flight at a 2D array of receiver pixels, e.g. time-of-flight cameras or flash lidar
• CPC Class Codes: G01S 7/4814 of transmitters alone
• CPC Class Codes: G01S 7/493 Extracting wanted echo signals
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/20024 Filtering details
• CPC Class Codes: G06T 2207/20192 Edge enhancement; Edge preservation
• CPC Class Codes: G06T 5/20 using local operators
• CPC Class Codes: G06T 5/70 Denoising; Smoothing
• CPC Class Codes: H04N 13/122 Improving the 3D impression of stereoscopic images by modifying image signal contents, e.g. by filtering or adding monoscopic depth cues H04N13/128 takes precedence
• CPC Class Codes: H04N 13/254 in combination with electromagnetic radiation sources for illuminating objects
• CPC Class Codes: H04N 25/60 Noise processing, e.g. detecting, correcting, reducing or removing noise

Abstract

A method for facilitating removal of multipath signal interference from light data can comprise illuminating, with an illumination unit, a target with a light source. The illumination unit can be configured to project a high spatial-frequency pattern onto the target in such a way as to redistribute spectral energy to higher frequencies. The method can also comprise acquiring, with a sensor unit, reflected light data reflected from the target. The reflected light data can comprise an array of spatial domain information received from light reflected by the target. Further, the method can comprise processing, with the one or more processors, the reflected light data. The processing applies a high-pass filter within the spatial domain to the reflected light data.

Claims

1. A time-of-flight camera apparatus configured to facilitate removal of multipath signal interference from light data, the apparatus comprising:

an illumination unit for illuminating a target with a light source, wherein;

the illumination unit is configured to project a high spatial-frequency pattern onto the target in such a way as to redistribute spectral energy to higher frequencies;

a sensor unit for acquiring reflected light data reflected from the target, wherein the reflected light data comprises an array of spatial domain information received from light reflected by the target; and

a processing unit for processing the reflected light data, wherein;

the processing unit applies a transformation that isolates a subset of high spatial frequencies in one or more directions within the reflected light data.

2. The time-of-flight camera apparatus as recited in claim 1, wherein the transformation is a high-pass or band-pass filter in one or more dimensions.

3. The time-of-flight camera apparatus as recited in claim 2, wherein the processing unit is further configured to:

generate multipath-compensated light data by applying the high-pass or band-pass filter within the spatial domain to the reflected light data, and

denoise the multipath-compensated light data with the reflected light data.

4. The time-of-flight camera apparatus as recited in claim 3, wherein the multipath-compensated light data comprises modulo 180°

phase data.

5. The time-of-flight camera apparatus as recited in claim 2, wherein the processing unit is further configured to:

generate multipath-compensated light data by applying the high-pass filter within the spatial domain to the reflected light data, and

the multipath-compensated light data is generated from a single frame acquired by the sensor unit with a single illumination pattern.

6. The time-of-flight camera apparatus as recited in claim 2, wherein frequency characteristics of the high-pass or band-pass filter are specifically optimized for the high spatial-frequency pattern.

7. The time-of-flight camera apparatus as recited in claim 1, wherein the high spatial-frequency pattern comprises a line light pattern.

8. The time-of-flight camera apparatus as recited in claim 1, wherein the high spatial-frequency pattern comprises a pattern of dots.

9. The time-of-flight camera apparatus as recited in claim 8, wherein the high spatial frequency isolating transform is implemented using a dot detection algorithm or intensity thresholding algorithm with the intent of ignoring multipath between the dots.

10. The time-of-flight camera apparatus as recited in claim 1, wherein the processing unit is configured to apply different high spatial-frequency patterns to subsequent frames.

11. The time-of-flight camera apparatus as recited in claim 1, wherein a first high spatial-frequency pattern comprises a line light pattern with the lines extending in a first direction and a second high spatial-frequency pattern comprises a line light pattern with the lines extending in a second direction.

12. The time-of-flight camera apparatus as recited in claim 1, wherein the processing unit is further configured to:

generate multipath-compensated light data by applying a filter within the spatial domain to the reflected light data which computes a regression curve between data components of spatial neighborhoods, such that regression parameters associated with the regression curve are suitable to compute multipath light data, and

denoise the multipath-compensated light data with the reflected light data.

13. A computer system for facilitating removal of multipath signal interference from light data, comprising:

one or more processors; and

one or more computer-readable media having stored thereon executable instructions that when executed by the one or more processors configure the computer system to perform at least the following;

illuminate, with an illumination unit, a target with a light source, wherein the illumination unit is configured to project a high spatial-frequency pattern onto the target in such a way as to redistribute spectral energy to higher frequencies;

acquire, with a sensor unit, reflected light data reflected from the target, wherein the reflected light data comprises;

an array of spatial domain information received from light reflected by the target, and

time-of-flight data for light that illuminated the target;

generate multipath-compensated light data by processing, with the one or more processors, the reflected light data, wherein the processing applies a high-pass filter within the spatial domain to the reflected light data; and

determine, with the one or more processors and based upon the multipath-compensated light data, one or more distances that extend between the sensor unit and one or more portions of the target.

14. The computer system as recited in claim 13, wherein the processing unit is configured to apply different high spatial-frequency patterns to concurrent frames.

15. The computer system as recited in claim 13, wherein a first high spatial-frequency pattern comprises a line light pattern with the lines extending in a first direction and a second high spatial-frequency pattern comprises a line light pattern with the lines extending in a second direction.

16. The computer system as recited in claim 13, wherein the processing unit is further configured to denoise the multipath-compensated light data with the reflected light data.

17. The computer system as recited in claim 16, wherein the multipath-compensated light data comprises modulo 180°

phase data.

18. The computer system as recited in claim 13, wherein the multipath-compensated light data is generated from a single frame acquired by the sensor unit using a single illumination pattern.

19. The computer system as recited in claim 13, wherein the time-of-flight camera apparatus comprises a Lidar.

20. A method, implemented at a computer system that includes one or more processors, for facilitating removal of multipath signal interference from light data, the method comprising:

illuminating, with an illumination unit, a target with a light source, wherein the illumination unit is configured to project a high spatial-frequency pattern onto the target in such a way as to redistribute spectral energy to higher frequencies;

acquiring, with a sensor unit, reflected light data reflected from the target, wherein the reflected light data comprises an array of spatial domain information received from light reflected by the target; and

processing, with the one or more processors, the reflected light data, wherein the processing applies a high-pass filter within the spatial domain to the reflected light data.