SPECULAR REFLECTION REMOVAL IN TIME-OF-FLIGHT CAMERA APPARATUS

• US 20170322309A1
• Filed: 05/09/2016
• Published: 11/09/2017
• Est. Priority Date: 05/09/2016
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Mirko Schmidt
• Inventors: Godbaz, John Peter, Bamji, Cyrus S., Schmidt, Mirko
• Granted Patent: US 10,234,561 B2
• Time in Patent Office: Days
• CPC Class Codes: G01S 17/08 for measuring distance only indirect measurement G01S17/46; active triangulation systems G01S17/48
• CPC Class Codes: G01S 17/50 Systems of measurement based on relative movement of target
• CPC Class Codes: G01S 17/894 3D imaging with simultaneous measurement of time-of-flight at a 2D array of receiver pixels, e.g. time-of-flight cameras or flash lidar
• CPC Class Codes: G01S 7/4814 of transmitters alone
• CPC Class Codes: G01S 7/493 Extracting wanted echo signals
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/20024 Filtering details
• CPC Class Codes: G06T 2207/20192 Edge enhancement; Edge preservation
• CPC Class Codes: G06T 5/20 using local operators
• CPC Class Codes: G06T 5/70 Denoising; Smoothing
• CPC Class Codes: H04N 13/122 Improving the 3D impression of stereoscopic images by modifying image signal contents, e.g. by filtering or adding monoscopic depth cues H04N13/128 takes precedence
• CPC Class Codes: H04N 13/254 in combination with electromagnetic radiation sources for illuminating objects
• CPC Class Codes: H04N 25/60 Noise processing, e.g. detecting, correcting, reducing or removing noise

Abstract

A method for facilitating removal of specular reflection noise from light data can include illuminating, using an illumination unit, a target with a light source. The illumination unit is configured to project light with a spatial light pattern onto the target. The method can also include acquiring, with a sensor unit, light data that is reflected from the target. The light data may comprise a directly reflected spatial light pattern and a specular reflected spatial light pattern. The directly reflected spatial light pattern and the specular reflected spatial light pattern comprise at least one spatial distinction that distinguishes the directly reflected spatial light pattern from the specular reflected spatial light pattern. The method can further comprise processing the light data to distinguish the directly reflected spatial light pattern from the specular reflected spatial light pattern based upon the at least one spatial distinction.

Claims

1. A time-of-flight camera apparatus configured to facilitate removal of specular reflection noise from light data, the apparatus comprising:

an illumination unit for illuminating a target with a light source, wherein;

the illumination unit is configured to project light with at least one spatial light pattern onto the target;

a sensor unit for acquiring light data that is reflected from the target, wherein;

the light data comprises a directly reflected spatial light pattern and a specular reflected spatial light pattern, and

the directly reflected spatial light pattern and the specular reflected spatial light pattern comprise at least one spatial distinction that distinguishes the directly reflected spatial light pattern from the specular reflected spatial light pattern; and

a processing unit for processing the light data, wherein;

the processing unit is configured to distinguish the directly reflected spatial light pattern from the specular reflected spatial light pattern based upon the at least one spatial distinction.

2. The time-of-flight camera apparatus as recited in claim 1, wherein the spatial light pattern corresponds with the directly reflected spatial light pattern.

3. The time-of-flight camera apparatus as recited in claim 1, wherein the directly reflected spatial light pattern comprises parallel lines extending in a first direction.

4. The time-of-flight camera apparatus as recited in claim 3, wherein:

the specular reflected spatial light pattern comprises parallel lines extending in a second direction; and

the at least one spatial distinction comprises the second direction being different than the first direction.

5. The time-of-flight camera apparatus as recited in claim 4, wherein the processing unit is configured to distinguish the directly reflected spatial light pattern from the specular reflected spatial light pattern by applying at least one directional filter.

6. The time-of-flight camera apparatus as recited in claim 5, wherein the at least one directional filter includes at least two directional filters, the at least two directional filters including a high-pass filter and a low pass filter.

7. The time-of-flight camera apparatus as recited in claim 1, wherein the at least one spatial light pattern is a pattern of dots.

8. The time-of-flight camera apparatus as recited in claim 6, wherein a dot detection or intensity thresholding algorithm is applied to the data.

9. The time-of-flight camera apparatus as recited in claim 6, wherein a comparison to a known spatial relationship between dots in the directly reflected spatial light is used to isolate directly reflected spatial light from specular reflected spatial light.

10. The time-of-flight camera apparatus as recited in claim 1, wherein the at least one spatial distinction used to isolate directly reflected spatial light from specular reflected spatial light is a relative time-of-flight.

11. The time-of-flight camera apparatus as recited in claim 1, wherein the time-of-flight camera utilizes a processing algorithm to distinguish between the at least one spatial light pattern and a set of affine transformations of the at least one spatial light pattern.

12. The time-of-flight camera apparatus as recited in claim 1, wherein the time-of-flight camera apparatus comprises a Lidar.

13. A computer system for facilitating removal of specular reflection noise from light data, comprising:

one or more processors; and

one or more computer-readable media having stored thereon executable instructions that when executed by the one or more processors configure the computer system to perform at least the following;

illuminate, using an illumination unit, a target with a light source, wherein the illumination unit is configured to project light with a spatial light pattern onto the target;

acquire, with a sensor unit, light data that is reflected from the target, wherein the light data comprises;

time-of-flight data for light that illuminated the target,

a directly reflected spatial light pattern and a specular reflected spatial light pattern, and

the directly reflected spatial light pattern and the specular reflected spatial light pattern comprise at least one spatial distinction that distinguishes the directly reflected spatial light pattern from the specular reflected spatial light pattern;

generate specular-reflection-compensated light data by processing, with the one or more processors, the light data, wherein the processing unit is configured to distinguish the directly reflected spatial light pattern from the specular reflected spatial light pattern based upon the at least one spatial distinction; and

determine, with the one or more processors and based upon the specular-reflection-compensated light data, one or more distances that extend between the light source and one or more portions of the target.

14. The computer system as recited in claim 13, wherein the spatial light pattern corresponds with the directly reflected spatial light pattern.

15. The computer system as recited in claim 13, wherein the directly reflected spatial light pattern comprises parallel lines extending in a first direction.

16. The computer system as recited in claim 15, wherein:

the specular reflected spatial light pattern comprises parallel lines extending in a second direction; and

the at least one spatial distinction comprises the second direction being different than the first direction.

17. The computer system as recited in claim 16, wherein the processing unit is configured to distinguish the directly reflected spatial light pattern from the specular reflected spatial light pattern by applying at least one directional filter.

18. The computer system as recited in claim 13, wherein a processing algorithm is used to distinguish between the spatial light pattern and a set of affine transformations of the spatial light pattern.

19. The computer system as recited in claim 13, wherein the at least one spatial distinction used to isolate directly reflected spatial light from specular reflected light is a relative time-of-flight.

20. A method, implemented at a computer system that includes one or more processors, for facilitating removal of specular reflection noise from light data, the method comprising:

illuminating, using an illumination unit, a target with a light source, wherein the illumination unit is configured to project light with a spatial light pattern onto the target;

acquiring, with a sensor unit, light data that is reflected from the target, wherein;

the light data comprises a directly reflected spatial light pattern and a specular reflected spatial light pattern, and

the directly reflected spatial light pattern and the specular reflected spatial light pattern comprise at least one spatial distinction that distinguishes the directly reflected spatial light pattern from the specular reflected spatial light pattern; and

processing, with a processing unit, the light data, wherein the processing unit is configured to distinguish the directly reflected spatial light pattern from the specular reflected spatial light pattern based upon the at least one spatial distinction.