Selective diffusion of filtered edges in images

• US 8,478,064 B2
• Filed: 01/19/2011
• Issued: 07/02/2013
• Est. Priority Date: 01/19/2010
• Status: Active Grant

• Current Assignee: Pixar (The Walt Disney Company)
• Original Assignee: Pixar (The Walt Disney Company)
• Inventors: Kass, Michael
• Primary Examiner(s): Do, Anh
• Application Number:
  US13/009,805
• Publication Number:
  US 20110229029A1
• Time in Patent Office: 895 Days
• Field of Search: 382/132, 382/128, 382/130, 382/199, 382/168, 382/260, 382/263, 382/264, 382/255, 356/218, 250/208.1, 250/214.AL, 128/925
• US Class Current: 382/264
• CPC Class Codes: G06T 2207/20012 Locally adaptive
• CPC Class Codes: G06T 2207/20028 Bilateral filtering
• CPC Class Codes: G06T 5/20 using local operators
• CPC Class Codes: G06T 5/70 Denoising; Smoothing

Abstract

An edge-preserving diffusion filter maintains the sharp edges in images while smoothing out image noise. An edge-preserving diffusion filter applies an edge-preserving smoothing filter to an image to form a filtered image. The modified image is then blurred by a blurring filter to form a blurred image. The modified image and the blurred image are blended together to form an output image based on an error metric associated with each pixel. The edge-preserving diffusion filter may be utilized to perform a multilevel decomposition of the image. The edge-preserving diffusion filter may be applied to an unfiltered image to produce a base image. The difference between the unfiltered image and the base image defines a detail image. The detail image may be used as the input for recursively generating additional levels of detail. The multilevel decomposition may utilize filter kernels associated with different contrast levels for each iteration.

Claims

1. A method of analyzing an image, the method comprising:

receiving, at one or more computer systems, at least one unfiltered image and at least one corresponding filtered image;

receiving, at the one or more computer systems, information indicative of a selection of a first image element of the filtered image;

receiving, at the one or more computer systems, information indicative of a first blurring operator;

applying, with one or more processors associated with the one or more computer systems, the first blurring operator to a neighborhood surrounding the first image element to determine a blurred image element, wherein the blurring operator is adapted to reduce a magnitude of a high spatial frequency component of an image spectrum relative to a magnitude of a low spatial frequency component of the image spectrum;

evaluating, with the one or more processors associated with the one or more computer systems, an error metric to compare the blurred image element and the first image element with a corresponding image element of the unfiltered image;

generating, with the one or more processors associated with the one or more computer systems, an output image including at least a first output image element; and

in response to the error metric satisfying a first criteria, setting, with the one or more processors associated with the one or more computer systems, the first output image element to equal the blurred image element.

2. The method of claim 1, wherein the error metric is adapted to quantify deviations of image elements from corresponding image elements in the unfiltered image.

3. The method of claim 1, wherein:

in response to the error metric satisfying a second criteria, setting, with the one or more processors associated with the one or more computer systems, the output image element to equal the first image element.

4. The method of claim 1, wherein:

in response to the error metric satisfying a second criteria, blending, with the one or more processors associated with the one or more computer systems, the first image element with the blurred image element to form a blended image element and setting the first output image element equal to the blended image element.

5. The method of claim 1, wherein the blurring operator is a linear filter kernel.

6. The method of claim 1, wherein the first blurring operator is selected from a plurality of blurring operators.

7. The method of claim 6, wherein the plurality of blurring operators including a Gaussian blurring operator.

8. The method of claim 1, further comprising:

receiving, at the one or more computer systems, information indicative of a selection of a second blurring operator;

applying, with the one or more processors associated with the one or more computer systems, the second blurring operator to a second neighborhood surrounding the first output image element in the output image to determine a second blurred image element;

evaluating, with the one or more processors associated with the one or more computer systems, a second error metric to compare the second blurred image element and the first image element with the first output image element in the output image;

generating, with the one or more processors associated with the one or more computer systems, a second output image including at least a second output image element; and

in response to the error metric satisfying the first criteria, setting, with the one or more processors associated with the one or more computer systems, the second output image element to equal the second blurred image element.

9. The method of claim 1, wherein the first and second blurring operators have different neighborhood sizes.

10. The method of claim 1, wherein the filtered image is the result of the application of a local histogram-based filter.

11. The method of claim 1, wherein the filtered image is the result of the application of a bilateral filter.

12. The method of claim 1, wherein the error metric is based on a ratio of differences between each of the blurred image element and the first image element with respect to the corresponding image element.

13. The method of claim 1, wherein the unfiltered image is included in a video sequence.

14. A method of analyzing an image, the method comprising:

receiving, at one or more computer systems, at least one unfiltered image;

applying, with one or more processors associated with the one or more computer systems, a first edge-preserving diffusion filter to the unfiltered image to form a first base image;

generating, with the one or more processors associated with the one or more computer systems, a first detail image based on differences between the unfiltered image and the first base image; and

storing the first base image and at least one detail image in a storage device associated with the one or more computer systems.

15. The method of claim 14, the method comprising:

applying, with the one or more processors associated with the one or more computer systems, a second edge-preserving diffusion filter to the first detail image to form a second base image; and

generating, with the one or more processors associated with the one or more computer systems, a second detail image based on differences between the unfiltered image and the second base image.

16. The method of claim 15, wherein the at least one detail image includes the second detail image.

17. The method of claim 15, wherein the first edge-preserving diffusion filter is applied with a first filter kernel associated with a first contrast level and the second edge-preserving diffusion filter is applied with a second filter kernel associated with a second contrast level.

18. The method of claim 17, wherein the first and second filter kernels are associated with a first neighborhood size.

19. The method of claim 17, wherein the at least one of the first and second filter kernels is a Gaussian filter kernel.

20. The method of claim 15, wherein at least one of the first and second edge-preserving diffusion filters includes an application of a local histogram-based filter.

21. The method of claim 15, wherein at least one of the first and second edge-preserving diffusion filters includes an application of a bilateral filter.

22. A method of analyzing an image, the method comprising:

receiving, at one or more computer systems, a first image;

applying, with one or more processors associated with the one or more computer systems, a first filter to the first image to create a filtered image;

applying, with the one or more processors associated with the one or more computer systems, a first blurring operator to the filtered image to create a blurred image;

comparing, with the one or more processors associated with the one or more computer systems, each of the filtered image and the blurred image with the first image to identify at least a first portion of the blurred image that is closer to the first image than the filtered image and at least a second portion of the filtered image that is closer to the first image than the blurred image; and

creating, with the one or more processors associated with the one or more computer systems, an output image including the first portion of the blurred image and the second portion of the filtered image.

23. The method of claim 22, wherein the blurring operator is adapted to reduce a magnitude of a high spatial frequency component of an image spectrum relative to a magnitude of a low spatial frequency component of the image spectrum.

24. The method of claim 22, wherein comparing each of the filtered image and the blurred image with the first image comprises:

determining, with the one or more processors associated with the one or more computer systems, a first error metric value for a first image element included in the blurred image based on a difference between the first image element and a corresponding second image element included in the first image;

determining, with the one or more processors associated with the one or more computer systems, a second error metric value for a third image element included in the filtered image based on a difference between the third image element and the second image element included in the first image, wherein the first image element in the blurred image corresponds with the second image element included in the filtered image;

determining, with the one or more processors associated with the one or more computer systems, if the first image element or the third image element is closer to the second image element in the first image based on the first and second error metric values;

including, with the one or more processors associated with the one or more computer systems, the first image element in the first portion of the blurred image in response to the first image element being closer to the second image element in the first image; and

including, with the one or more processors associated with the one or more computer systems, the third image element in the second portion of the filtered image in response to the third image element being closer to the second image element in the first image.

25. The method of claim 22, comprising:

identifying, with the one or more processors associated with the one or more computer systems, a third portion of the blurred image and a fourth portion of the filtered image having similar deviations from the first image;

blending, with the one or more processors associated with the one or more computer systems, the third portion of the blurred image with the fourth portion of the filtered image to form a blended image portion; and

including, with the one or more processors associated with the one or more computer systems, the blended image portion in the output image.

26. The method of claim 22, wherein the first blurring operator includes a Gaussian blurring operator.

27. The method of claim 22, further comprising:

applying, with the one or more processors associated with the one or more computer systems, a second blurring operator to the output image to create a second blurred image;

comparing, with the one or more processors associated with the one or more computer systems, each of the output image and the second blurred image with the first image to identify at least a third portion of the second blurred image that is closer to the first image than the output image and a fourth portion of the output image that is closer to the first image than the second blurred image;

creating, with the one or more processors associated with the one or more computer systems, a second output image including the third portion of the second blurred image and the fourth portion of the output image.

28. The method of claim 27, wherein the first blurring operator is associated with a first neighborhood having a first size and the second blurring operator is associated with a second neighborhood having a second size different than the first size.

29. The method of claim 22, wherein the first filter includes a local histogram-based filter.

30. The method of claim 22, wherein the first filter includes a bilateral filter.