Morphological and geometric edge filters for edge enhancement in depth images

• US 9,852,495 B2
• Filed: 12/22/2015
• Issued: 12/26/2017
• Est. Priority Date: 12/22/2015
• Status: Expired due to Fees

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Silver, David H., Bronstein, Michael, Bronstein, Alex, Kimmel, Ron, Sperling, Erez, Surazhsky, Vitaly, Zabatani, Aviad, Manashe, Ohad
• Primary Examiner(s): Moyer, Andrew
• Application Number:
  US14/978,429
• Publication Number:
  US 20170178305A1
• Time in Patent Office: 735 Days
• CPC Class Codes: G06T 2207/20036 Morphological image processing
• CPC Class Codes: G06T 2207/20192 Edge enhancement; Edge preservation
• CPC Class Codes: G06T 3/403 Edge-driven scaling; Edge-based scaling
• CPC Class Codes: G06T 5/70 Denoising; Smoothing

Abstract

A mechanism is described for facilitating three-dimensional (3D) depth imaging systems, and morphological and geometric filters for edge enhancement in depth images at computing devices according to one embodiment. A method of embodiments, as described herein, includes detecting an input digital image of an object, the digital image comprising data pixels contaminated by noise and confidence values corresponding to the data pixels, and computing a morphological filter by matching the confidence pixels in the input digital image with a set of matching templates, and using a set of masking templates to determine the data pixels and confidence pixels in the filtered image. The method further include computing an edge filter by performing computation of distances between the data pixels along a plurality of directions to determine an edge direction, and determining the data pixels and and the confidence pixels in a filtered image based on the edge direction. The method may further include applying at least one of the morphological filter and the edge filter to filter the digital image.

Claims

1. An apparatus comprising:

detection/reception logic to detect an input digital image of an object, the digital image comprising data pixels contaminated by noise and confidence pixels corresponding to the data pixels;

morphological filter logic of filter computation engine to compute a morphological filter by matching the confidence pixels in the input digital image with a set of matching templates,

and using a set of masking templates to determine the data pixels and confidence pixels in the filtered image;

edge filter logic of the filter computation engine to compute an edge filter by performing computation of distances between the data pixels along a plurality of directions to determine an edge direction, and determining the data pixels and the confidence pixels in a filtered image based on the edge direction; and

application/execution logic to apply at least one of the morphological filter and the edge filter to filter the digital image,

wherein the morphological and edge filters are applied by running a sliding window on the input digital image and computing a filter output for each position of the sliding window, wherein the sliding window represents a block of the data pixels and the confidence pixels extracted from the input digital image for each position, wherein the confidence pixel values provide accuracy of an estimation of corresponding data pixels,

wherein computing the morphological filter comprises measuring a distance between a window of the confidence pixels and the set of matching templates to determine a best matching template, determining an output confidence value corresponding to a window location based on a center pixel of a best matching template of the set of matching templates, and determining an output data value corresponding to the window location using a masking template corresponding to the best matching template applied to the data values in the window,

wherein the distance between the window of the confidence pixels and the matching template is a sum of absolute differences or a sum of squared differences between corresponding confidence pixel values in the window and the matching template, and

wherein the confidence pixel values and the masking template pixel values are binary, the distance between the confidence pixels and the matching template is the Hamming distance, and computing the aggregate comprises computing an AND binary function of the corresponding confidence pixel values and the masking template pixel values, and outputting a median value of the data pixel values at positions where an output of the AND binary function is equal to 1.

2. The apparatus of claim 1, further comprising communication and/or compatibility logic

to output the filtered digital image, wherein the output comprises the data pixels and the confidence pixels.

3. The apparatus of claim 1, wherein the digital image is captured via one or more

sensing components of an imaging system at the apparatus, wherein the one or more sensing components include a depth-sensing camera to capture the digital image of the object including a three-dimensional (3D) object, wherein the digital image is a depth image, wherein the noise further comprises at least one of depth quantization noise, saw-tooth noise, Gaussian noise, Poisson noise, salt-and-pepper noise, short noise, anisotropic noise, and film grain.

4. The apparatus of claim 1, wherein determining the output data value comprises

computing an aggregate of the data pixel values in the window, weighted by the corresponding confidence pixel values in the window and the corresponding pixels values in the masking template, and

wherein the aggregate is one of a weighted arithmetic mean, a weighted harmonic mean, a weighted geometric mean, and a weighted median.

5. The apparatus of claim 1, wherein the plurality of directions corresponds to

directions at a plurality of angles around the center pixel of the sliding window,

wherein the plurality of angles comprise four angles of O degrees, 45 degrees, 90 degrees, and 135 degrees,

wherein one or more of the directions having low confidence pixels are ignored at end points,

wherein determining the edge direction comprises computing an absolute difference of the data pixels at an end point of each direction, computing a maximum absolute difference over the directions to determine a maximum absolute value direction, computing a minimum absolute difference over the directions to determine a minimum absolute value direction, comparing the maximum absolute difference to a maximum threshold value, or comparing the minimum absolute difference to a minimum threshold value,

wherein computing the output data pixel for each sliding window position comprises

computing a mean of the data pixel values at an end points of a minimum absolute value direction, if the minimum absolute difference is below the minimum threshold value, and if the maximum absolute difference is above the maximum threshold value, or computing the aggregate of the data pixels values weighted by the corresponding confidence pixel values, and

wherein the aggregate is one of the weighted arithmetic mean, the weighted harmonic

mean, the weighted geometric mean, and the weighted median.

6. A method comprising:

detecting an input digital image of an object, the digital image comprising data pixels

contaminated by noise and confidence pixels corresponding to the data pixels;

computing a morphological filter by matching the confidence pixels in the input digital image with a set of matching templates, and using a set of masking templates to determine the data pixels and confidence pixels in the filtered image;

computing an edge filter by performing computation of distances between the data pixels along a plurality of directions to determine an edge direction, and determining the data pixels and the confidence pixels in a filtered image based on the edge direction; and

applying at least one of the morphological filter and the edge filter to filter the digital image,

wherein the morphological and edge filters are applied by running a sliding window on the input digital image and computing a filter output for each position of the sliding window, wherein the sliding window represents a block of the data pixels and the confidence pixels extracted from the input digital image for each position, wherein the confidence pixel values provide accuracy of an estimation of corresponding data pixels,

wherein computing the morphological filter comprises measuring a distance between a window of the confidence pixels and the set of matching templates to determine a best matching template, determining an output confidence value corresponding to a window location based on a center pixel of a best matching template of the set of matching templates, and determining an output data value corresponding to the window location using a masking template corresponding to the best matching template applied to the data values in the window,

wherein the distance between the window of the confidence pixels and the matching template is a sum of absolute differences or a sum of squared differences between corresponding confidence pixel values in the window and the matching template, and

wherein the confidence pixel values and the masking template pixel values are binary, the distance between the confidence pixels and the matching template is the Hamming distance, and computing the aggregate comprises computing an AND binary function of the corresponding confidence pixel values and the masking template pixel values, and outputting a median value of the data pixel values at positions where an output of the AND binary function is equal to 1.

7. The method of claim 6, further comprising outputting the filtered digital image,

wherein the output comprises the data pixels and the confidence pixels.

8. The method of claim 6, wherein the digital image is captured via one or more

capturing/sensing components of an imaging system at the apparatus, wherein the one or more capturing/sensing components include a depth-sensing camera to capture the digital image of the object including a three-dimensional (3D) object, wherein the digital image is a depth image,

wherein the noise further comprises at least one of depth quantization noise, saw-tooth noise, Gaussian noise, Poisson noise, salt-and-pepper noise, short noise, anisotropic noise, and film grain.

9. The method of claim 6, wherein determining the output data value comprises

computing an aggregate of the data pixel values in the window, weighted by the corresponding confidence pixel values in the window and the corresponding pixels values in the masking template, and

wherein the aggregate is one of a weighted arithmetic mean, a weighted harmonic mean, a weighted geometric mean, and a weighted median.

10. The method of claim 6, wherein the plurality of directions corresponds to

directions at a plurality of angles around the center pixel of the sliding window, wherein the plurality of angles comprise four angles of O degrees, 45 degrees, 90 degrees, and 135 degrees,

wherein one or more of the directions having low confidence pixels are ignored at end points,

wherein determining the edge direction comprises computing an absolute difference of the data pixels at an end point of each direction, computing a maximum absolute difference over the directions to determine a maximum absolute value direction, computing a minimum absolute difference over the directions to determine a minimum absolute value direction, comparing the maximum absolute difference to a maximum threshold value, or comparing the minimum absolute difference to a minimum threshold value,

wherein computing the output data pixel for each sliding window position comprises

computing a mean of the data pixel values at an end points of a minimum absolute value direction, if the minimum absolute difference is below the minimum threshold value, and if the maximum absolute difference is above the maximum threshold value, or computing the aggregate of the data pixels values weighted by the corresponding confidence pixel values, and

wherein the aggregate is one of the weighted arithmetic mean, the weighted harmonic mean, the weighted geometric mean, and the weighted median.

11. At least one non-transitory machine-readable medium comprising a plurality of instructions,

executed on a computing device, to facilitate the computing device to perform operations comprising:

detecting an input digital image of an object, the digital image comprising data pixels

contaminated by noise and confidence pixels corresponding to the data pixels;

computing a morphological filter by matching the confidence pixels in the input digital image with a set of matching templates, and using a set of masking templates to determine the data pixels and confidence pixels in the filtered image;

computing an edge filter by performing computation of distances between the data pixels along a plurality of directions to determine an edge direction, and determining the data pixels and the confidence pixels in a filtered image based on the edge direction; and

applying at least one of the morphological filter and the edge filter to filter the digital image,

wherein the morphological and edge filters are applied by running a sliding window on the input digital image and computing a filter output for each position of the sliding window, wherein the sliding window represents a block of the data pixels and the confidence pixels extracted from the input digital image for each position, wherein the confidence pixel values provide accuracy of an estimation of corresponding data pixels,

wherein computing the morphological filter comprises measuring a distance between a window of the confidence pixels and the set of matching templates to determine a best matching template, determining an output confidence value corresponding to a window location based on a center pixel of a best matching template of the set of matching templates, and determining an output data value corresponding to the window location using a masking template corresponding to the best matching template applied to the data values in the window,

wherein the distance between the window of the confidence pixels and the matching template is a sum of absolute differences or a sum of squared differences between corresponding confidence pixel values in the window and the matching template, and

wherein the confidence pixel values and the masking template pixel values are binary, the distance between the confidence pixels and the matching template is the Hamming distance, and computing the aggregate comprises computing an AND binary function of the corresponding confidence pixel values and the masking template pixel values, and outputting a median value of the data pixel values at positions where an output of the AND binary function is equal to 1.

12. The non-transitory machine-readable medium of claim 11, wherein the operations further

comprise outputting the filtered digital image, wherein the output comprises the data pixels and the confidence pixels,

wherein the digital image is captured via one or more capturing/sensing components of an imaging system at the apparatus, wherein the one or more capturing/sensing components include a depth-sensing camera to capture the digital image of the object including a three-dimensional (3D) object, wherein the digital image is a depth image, wherein the noise further comprises at least one of depth quantization noise, saw-tooth noise, Gaussian noise, Poisson noise, salt-and-pepper noise, short noise, anisotropic noise, and film grain,

wherein the morphological and edge filters are applied by running a sliding window on the input digital image and computing a filter output for each position of the sliding window,

wherein the sliding window represents a block of the data pixels and the confidence pixels extracted from the input digital image for each position, wherein the confidence pixel values provide accuracy of an estimation of corresponding data pixels.

13. The non-transitory machine-readable medium of claim 11, wherein determining the output data

value comprises computing an aggregate of the data pixel values in the window, weighted by the corresponding confidence pixel values in the window and the corresponding pixels values in the masking template, and

wherein the aggregate is one of a weighted arithmetic mean, a weighted harmonic mean, a weighted geometric mean, and a weighted median.

14. The non-transitory machine-readable medium of claim 11, wherein the plurality of directions

corresponds to directions at a plurality of angles around the center pixel of the sliding window,

wherein the plurality of angles comprise four angles of O degrees, 45 degrees, 90 degrees, and 135 degrees, wherein one or more of the directions having low confidence pixels are ignored at end points,

wherein determining the edge direction comprises computing an absolute difference of the data pixels at an end point of each direction, computing a maximum absolute difference over the directions to determine a maximum absolute value direction, computing a minimum absolute difference over the directions to determine a minimum absolute value direction, comparing the maximum absolute difference to a maximum threshold value, or comparing the minimum absolute difference to a minimum threshold value,

wherein computing the output data pixel for each sliding window position comprises

computing a mean of the data pixel values at an end points of a minimum absolute value direction, if the minimum absolute difference is below the minimum threshold value, and if the maximum absolute difference is above the maximum threshold value, or computing the aggregate of the data pixels values weighted by the corresponding confidence pixel values, and

wherein the aggregate is one of the weighted arithmetic mean, the weighted harmonic mean, the weighted geometric mean, and the weighted median.