MORPHOLOGICAL AND GEOMETRIC EDGE FILTERS FOR EDGE ENHANCEMENT IN DEPTH IMAGES

• US 20170178305A1
• Filed: 12/22/2015
• Published: 06/22/2017
• Est. Priority Date: 12/22/2015
• Status: Active Grant

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: SILVER, DAVID H., BRONSTEIN, ALEX, KIMMEL, RON, SPERLING, EREZ, SURAZHSKY, VITALY, ZABATANI, AVIAD, MANASHE, OHAD, BRONSTEIN, MICHAEL
• Granted Patent: US 9,852,495 B2
• Time in Patent Office: Days
• CPC Class Codes: G06T 2207/20036 Morphological image processing
• CPC Class Codes: G06T 2207/20192 Edge enhancement; Edge preservation
• CPC Class Codes: G06T 3/403 Edge-driven scaling; Edge-based scaling
• CPC Class Codes: G06T 5/70 Denoising; Smoothing

Abstract

A mechanism is described for facilitating three-dimensional (3D) depth imaging systems, and morphological and geometric filters for edge enhancement in depth images at computing devices according to one embodiment. A method of embodiments, as described herein, includes detecting an input digital image of an object, the digital image comprising data pixels contaminated by noise and confidence values corresponding to the data pixels, and computing a morphological filter by matching the confidence pixels in the input digital image with a set of matching templates, and using a set of masking templates to determine the data pixels and confidence pixels in the filtered image. The method further include computing an edge filter by performing computation of distances between the data pixels along a plurality of directions to determine an edge direction, and determining the data pixels and and the confidence pixels in a filtered image based on the edge direction. The method may further include applying at least one of the morphological filter and the edge filter to filter the digital image.

Claims

1. An apparatus comprising:

detection/reception logic to detect an input digital image of an object, the digital image comprising data pixels contaminated by noise and confidence pixels corresponding to the data pixels;

morphological filter logic of filter computation engine to compute a morphological filter by matching the confidence pixels in the input digital image with a set of matching templates, and using a set of masking templates to determine the data pixels and confidence pixels in the filtered image;

edge filter logic of the filter computation engine to compute an edge filter by performing computation of distances between the data pixels along a plurality of directions to determine an edge direction, and determining the data pixels and and the confidence pixels in a filtered image based on the edge direction; and

application/execution logic to apply at least one of the morphological filter and the edge filter to filter the digital image.

2. The apparatus of claim 1, further comprising communication/compatibility logic to output the filtered digital image, wherein the output comprises the data pixels and the confidence pixels.

3. The apparatus of claim 1, wherein the digital image is captured via one or more capturing/sensing components of an imaging system at the apparatus, wherein the one or more capturing/sensing components include a depth-sensing camera to capture the digital image of the object including a three-dimensional (3D) object, wherein the digital image is a depth image, wherein the noise further comprises at least one of depth quantization noise, saw-tooth noise, Gaussian noise, Poisson noise, salt-and-pepper noise, short noise, anisotropic noise, and film grain.

4. The apparatus of claim 1, wherein the morphological and edge filters are applied by running a sliding window on the input digital image and computing a filter output for each position of the sliding window, wherein the sliding window represents a block of the data pixels and the confidence pixels extracted from the input digital image for each position, wherein the confidence pixel values provide accuracy of an estimation of corresponding data pixels.

5. The apparatus of claim 4, wherein computing the morphological filter comprises:

measuring a distance between a window of the confidence pixels and the set of matching templates to determine a best matching template;

determining an output confidence value corresponding to a window location based on a center pixel of a best matching template of the set of matching templates; and

determining an output data value corresponding to the window location using a masking template corresponding to the best matching template applied to the data values in the window.

6. The apparatus of claim 5, wherein the distance between the window of the confidence pixels and the matching template is a sum of absolute differences or a sum of squared differences between corresponding confidence pixel values in the window and the matching template.

7. The apparatus of claim 6, wherein determining the output data value comprises computing an aggregate of the data pixel values in the window, weighted by the corresponding confidence pixel values in the window and the corresponding pixels values in the masking template, and

wherein the aggregate is one of a weighted arithmetic mean, a weighted harmonic mean, a weighted geometric mean, and a weighted median.

8. The apparatus of claim 6, wherein the confidence pixel values and the masking template pixel values are binary,

the distance between the confidence pixels and the matching template is the Hamming distance, and

computing the aggregate comprises computing an AND binary function of the corresponding confidence pixel values and the masking template pixel values, and outputting a median value of the data pixel values at positions where an output of the AND binary function is equal to 1.

9. The apparatus of claim 1, wherein the plurality of directions corresponds to directions at a plurality of angles around the center pixel of the sliding window, wherein the plurality of angles comprise four angles of 0 degrees, 45 degrees, 90 degrees, and 135 degrees, wherein one or more of the directions having low confidence pixels are ignored at end points,

wherein determining the edge direction comprises computing an absolute difference of the data pixels at an end point of each direction, computing a maximum absolute difference over the directions to determine a maximum absolute value direction, computing a minimum absolute difference over the directions to determine a minimum absolute value direction, comparing the maximum absolute difference to a maximum threshold value, or comparing the minimum absolute difference to a minimum threshold value,

wherein computing the output data pixel for each sliding window position comprises computing a mean of the data pixel values at an end points of a minimum absolute value direction, if the minimum absolute difference is below the minimum threshold value, and if the maximum absolute difference is above the maximum threshold value, or computing the aggregate of the data pixels values weighted by the corresponding confidence pixel values, and

wherein the aggregate is one of the weighted arithmetic mean, the weighted harmonic mean, the weighted geometric mean, and the weighted median.

10. A method comprising:

detecting an input digital image of an object, the digital image comprising data pixels contaminated by noise and confidence pixels corresponding to the data pixels;

computing a morphological filter by matching the confidence pixels in the input digital image with a set of matching templates, and using a set of masking templates to determine the data pixels and confidence pixels in the filtered image;

computing an edge filter by performing computation of distances between the data pixels along a plurality of directions to determine an edge direction, and determining the data pixels and and the confidence pixels in a filtered image based on the edge direction; and

applying at least one of the morphological filter and the edge filter to filter the digital image.

11. The method of claim 10, further comprising outputting the filtered digital image, wherein the output comprises the data pixels and the confidence pixels.

12. The method of claim 10, wherein the digital image is captured via one or more capturing/sensing components of an imaging system at the apparatus, wherein the one or more capturing/sensing components include a depth-sensing camera to capture the digital image of the object including a three-dimensional (3D) object, wherein the digital image is a depth image, wherein the noise further comprises at least one of depth quantization noise, saw-tooth noise, Gaussian noise, Poisson noise, salt-and-pepper noise, short noise, anisotropic noise, and film grain.

13. The method of claim 10, wherein the morphological and edge filters are applied by running a sliding window on the input digital image and computing a filter output for each position of the sliding window, wherein the sliding window represents a block of the data pixels and the confidence pixels extracted from the input digital image for each position, wherein the confidence pixel values provide accuracy of an estimation of corresponding data pixels.

14. The method of claim 13, wherein computing the morphological filter comprises:

measuring a distance between a window of the confidence pixels and the set of matching templates to determine a best matching template;

determining an output confidence value corresponding to a window location based on a center pixel of a best matching template of the set of matching templates; and

determining an output data value corresponding to the window location using a masking template corresponding to the best matching template applied to the data values in the window.

15. The method of claim 14, wherein the distance between the window of the confidence pixels and the matching template is a sum of absolute differences or a sum of squared differences between corresponding confidence pixel values in the window and the matching template.

16. The method of claim 15, wherein determining the output data value comprises computing an aggregate of the data pixel values in the window, weighted by the corresponding confidence pixel values in the window and the corresponding pixels values in the masking template, and

wherein the aggregate is one of a weighted arithmetic mean, a weighted harmonic mean, a weighted geometric mean, and a weighted median.

17. The method of claim 15, wherein the confidence pixel values and the masking template pixel values are binary,

the distance between the confidence pixels and the matching template is the Hamming distance, and

computing the aggregate comprises computing an AND binary function of the corresponding confidence pixel values and the masking template pixel values, and outputting a median value of the data pixel values at positions where an output of the AND binary function is equal to 1.

18. The method of claim 10, wherein the plurality of directions corresponds to directions at a plurality of angles around the center pixel of the sliding window, wherein the plurality of angles comprise four angles of 0 degrees, 45 degrees, 90 degrees, and 135 degrees, wherein one or more of the directions having low confidence pixels are ignored at end points,

wherein determining the edge direction comprises computing an absolute difference of the data pixels at an end point of each direction, computing a maximum absolute difference over the directions to determine a maximum absolute value direction, computing a minimum absolute difference over the directions to determine a minimum absolute value direction, comparing the maximum absolute difference to a maximum threshold value, or comparing the minimum absolute difference to a minimum threshold value,

wherein computing the output data pixel for each sliding window position comprises computing a mean of the data pixel values at an end points of a minimum absolute value direction, if the minimum absolute difference is below the minimum threshold value, and if the maximum absolute difference is above the maximum threshold value, or computing the aggregate of the data pixels values weighted by the corresponding confidence pixel values, and

wherein the aggregate is one of the weighted arithmetic mean, the weighted harmonic mean, the weighted geometric mean, and the weighted median.

19. At least one machine-readable medium comprising a plurality of instructions, executed on a computing device, to facilitate the computing device to perform operations comprising:

detecting an input digital image of an object, the digital image comprising data pixels contaminated by noise and confidence pixels corresponding to the data pixels;

computing a morphological filter by matching the confidence pixels in the input digital image with a set of matching templates, and using a set of masking templates to determine the data pixels and confidence pixels in the filtered image;

computing an edge filter by performing computation of distances between the data pixels along a plurality of directions to determine an edge direction, and determining the data pixels and and the confidence pixels in a filtered image based on the edge direction; and

applying at least one of the morphological filter and the edge filter to filter the digital image.

20. The machine-readable medium of claim 19, wherein the operations further comprise outputting the filtered digital image, wherein the output comprises the data pixels and the confidence pixels,

wherein the digital image is captured via one or more capturing/sensing components of an imaging system at the apparatus, wherein the one or more capturing/sensing components include a depth-sensing camera to capture the digital image of the object including a three-dimensional (3D) object, wherein the digital image is a depth image, wherein the noise further comprises at least one of depth quantization noise, saw-tooth noise, Gaussian noise, Poisson noise, salt-and-pepper noise, short noise, anisotropic noise, and film grain,

wherein the morphological and edge filters are applied by running a sliding window on the input digital image and computing a filter output for each position of the sliding window, wherein the sliding window represents a block of the data pixels and the confidence pixels extracted from the input digital image for each position, wherein the confidence pixel values provide accuracy of an estimation of corresponding data pixels.

21. The machine-readable medium of claim 20, wherein computing the morphological filter comprises:

measuring a distance between a window of the confidence pixels and the set of matching templates to determine a best matching template;

determining an output confidence value corresponding to a window location based on a center pixel of a best matching template of the set of matching templates; and

determining an output data value corresponding to the window location using a masking template corresponding to the best matching template applied to the data values in the window.

22. The machine-readable medium of claim 21, wherein the distance between the window of the confidence pixels and the matching template is a sum of absolute differences or a sum of squared differences between corresponding confidence pixel values in the window and the matching template.

23. The machine-readable medium of claim 22, wherein determining the output data value comprises computing an aggregate of the data pixel values in the window, weighted by the corresponding confidence pixel values in the window and the corresponding pixels values in the masking template, and

wherein the aggregate is one of a weighted arithmetic mean, a weighted harmonic mean, a weighted geometric mean, and a weighted median.

24. The machine-readable medium of claim 22, wherein the confidence pixel values and the masking template pixel values are binary,

the distance between the confidence pixels and the matching template is the Hamming distance, and

computing the aggregate comprises computing an AND binary function of the corresponding confidence pixel values and the masking template pixel values, and outputting a median value of the data pixel values at positions where an output of the AND binary function is equal to 1.

25. The machine-readable medium of claim 19, wherein the plurality of directions corresponds to directions at a plurality of angles around the center pixel of the sliding window, wherein the plurality of angles comprise four angles of 0 degrees, 45 degrees, 90 degrees, and 135 degrees, wherein one or more of the directions having low confidence pixels are ignored at end points,

wherein determining the edge direction comprises computing an absolute difference of the data pixels at an end point of each direction, computing a maximum absolute difference over the directions to determine a maximum absolute value direction, computing a minimum absolute difference over the directions to determine a minimum absolute value direction, comparing the maximum absolute difference to a maximum threshold value, or comparing the minimum absolute difference to a minimum threshold value,

wherein computing the output data pixel for each sliding window position comprises computing a mean of the data pixel values at an end points of a minimum absolute value direction, if the minimum absolute difference is below the minimum threshold value, and if the maximum absolute difference is above the maximum threshold value, or computing the aggregate of the data pixels values weighted by the corresponding confidence pixel values, and

wherein the aggregate is one of the weighted arithmetic mean, the weighted harmonic mean, the weighted geometric mean, and the weighted median.