System and method for video noise reduction using a unified three-dimensional non-linear filtering

• US 20080205854A1
• Filed: 02/23/2007
• Published: 08/28/2008
• Est. Priority Date: 02/23/2007
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Kim, Yeong-Taeg, Xu, Ning
• Granted Patent: US 8,063,995 B2
• Time in Patent Office: Days
• US Class Current: 386/114
• CPC Class Codes: H04N 19/117 Filters, e.g. for pre-processing or post-processing sub-band filter banks H04N19/635
• CPC Class Codes: H04N 19/137 Motion inside a coding unit, e.g. average field, frame or block difference
• CPC Class Codes: H04N 19/182 the unit being a pixel
• CPC Class Codes: H04N 19/80 Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation H04N19/635, H04N19/86 take precedence
• CPC Class Codes: H04N 19/85 using pre-processing or post-processing specially adapted for video compression

Abstract

The system described herein is a unified approach to three-dimensional filtering for the reduction of video noise. The technique is based on weight averaging pixels of the filter'"'"'s output value in a three-dimensional neighborhood, in space and time, of the filter'"'"'s output value. The weights can be computed for individual factors, such as distance, regional differences, etc., or can be combined into a weight that is indicative of all individual weights.

Claims

1. A method of filtering pixels in a plurality of video frames, the method comprising:

selecting a first pixel from one of the video frames, the first pixel having location coordinates at a horizontal position x, a vertical position y, and a time t;

determining weights for a set of pixel intensities in the video frames, wherein each weight is based on at least one pixel in a neighborhood of the first pixel and at least one pixel in a neighborhood of the pixel to be weighted;

determining a weighted average using the weights and the pixel intensities; and

identifying the weighted average as the filtered output at pixel location x, y, and t.

2. The method of claim 1, wherein the weight applied to a pixel is determined by a functional relationship to at least one of a group comprising the following metrics:

the spatial distance between the location coordinates x, y, and the spatial location coordinates of the pixel being averaged;

the spatial distance between the location coordinates x, y and the motion compensated values of the spatial location coordinates of the pixel being averaged;

the difference between the time t and a value of time associated with the pixel being averaged;

the average of the absolute value of difference between the average intensity in a two-dimensional spatial neighborhood of the pixel being averaged and the average intensity in a spatial neighborhood of the pixel at the x and y;

the absolute value of the average difference between the intensity of the pixel at x, y, t and the intensity of the pixel being averaged; and

the average, taken over a two-dimensional neighborhood of x and y, of the absolute value of the difference between the intensity of a pixel displaced from x and y and the intensity of a pixel identically displaced from the pixel being averaged.

3. The method of claim 2, wherein a group of weights is determined by applying a functional relationship defined for each weight to the metric to which it applies.

4. The method of claim 2, wherein an intermediate weight is determined as a multivariate function of the weights in the group of weights.

5. The method of claim 4, wherein a metric is determined, for each set of location coordinates, by ranking the intermediate weight in an ordering of such weights, and including its rank in the group of metrics.

6. The method of claim 5, wherein the weight determined by ranking the intermediate weights is unity if the rank is equal or higher than a predetermined constant and is zero if the rank is less than that constant.

7. The method of claim 6, wherein the weight applied to the pixel in the weighted average of pixels is the intermediate weight in correspondence to the weight determined by the ranking being unity and wherein the weight is zero otherwise.

8. The method of claim 4, wherein the multivariate function of the weights comprises multiplying the weights together.

9. The method of claim 3, wherein the functional relationship between the metrics and weights comprises four quantities, v₁, v₂, Th₁, and Th₂ and a piecewise linear input/output characteristic whose functional dependence is as follows:

if the input is less than Th₁, the output is v₁;

if the input is greater than Th₂, the output is v₂; and

if the input is between Th₁ and Th₂, the output is found by linearly interpolating between the points (Th₁, v₁) and (Th₂, v₂).

10. The method of claim 9, wherein the values of the four quantities are determined by simulation.

11. The method of claim 1, wherein the neighborhood has a dimension of from about 2 to about 5 frames by from about 2 to about 8 lines by from about 10 to about 20 pixels per line.

12. The method of claim 11, wherein the neighborhood has a dimension of 3 frames by 5 lines by 15 pixels per line.

13. A modular system for filtering pixels in a plurality of video frames, the modules comprising:

a data module configured to select a first pixel from one of the video frames, the first pixel having location coordinates at a horizontal position x, a vertical position y, and a time t;

a weight module configured to determine weights for a set of pixel intensities in the video frames, wherein each weight is based on at least one pixel in a neighborhood of the first pixel and at least one pixel in a neighborhood of the pixel to be weighted; and

a summation module configured to determine a weighted average using the weights and the pixel intensities that is identified as the filtered output at pixel location x, y, and t.

14. The modular system of claim 13, wherein the data module also produces the pixels and their location coordinates in the neighborhoods of the second pixel and the first pixel respectively.

15. The modular system of claim 14, wherein the weight module determines a group of metrics comprising:

the spatial distance between the coordinates x, y, and the location coordinates of the pixel being averaged;

the spatial distance between the spatial coordinates x, y and the motion compensated values of the spatial coordinates of the pixel being averaged;

the difference between the time t and a value of time associated with the pixel being averaged;

the absolute value of the average difference between the intensity of the pixel at x, y, t and the intensity of the pixel being averaged;

the absolute value of the difference between the average intensity in a two-dimensional spatial neighborhood of the pixel being averaged and the average intensity in a spatial neighborhood of the pixel at the x and y; and

the sum, taken over a two-dimensional neighborhood of x and y, of the absolute value of the difference between the intensity of a pixel displaced from x and y and the intensity of a pixel identically displaced from the pixel being averaged.

16. The modular system of claim 15, wherein a group of weights is determined by applying a functional relationship defined for each weight to the metric to which it applies.

17. The method of claim 13, wherein the neighborhood has a dimension of 3 frames by 5 lines by 15 pixels per line.

18. A machine readable medium comprising instructions that upon execution cause a machine to perform a method of filtering pixels in a plurality of video frames, the method comprising:

selecting a first pixel from one of the video frames, the first pixel having location coordinates at a horizontal position x, a vertical position y, and a time t;

determining weights for a set of pixel intensities in the video frames, wherein each weight is based on at least one pixel in a neighborhood of the first pixel and at least one pixel in a neighborhood of the pixel to be weighted;

determining a weighted average using the weights and the pixel intensities; and

identifying the weighted average as the filtered output at pixel location x, y, and t.

19. The machine readable medium of claim 18, wherein the weight applied to a pixel is determined by a functional relationship to at least one of a group comprising the following metrics:

the spatial distance between the location coordinates x, y, and t and the location coordinates of the pixel being averaged;

the spatial distance between the spatial coordinates x, y and the motion compensated values of the spatial coordinates of the pixel being averaged;

the difference between the time t and a value of time associated with the pixel being averaged;

the absolute value of the average difference between the intensity of the pixel at x, y, t and the intensity of the pixel being averaged;

the absolute value of the difference between the average intensity in a two-dimensional spatial neighborhood of the pixel being averaged and the average intensity in a spatial neighborhood of the pixel at the x and y; and

the sum, taken over a two-dimensional neighborhood of x and y, of the absolute value of the difference between the intensity of a pixel displaced from x and y and the intensity of a pixel identically displaced from the pixel being averaged.

20. The machine readable medium of claim 19, wherein a group of weights is determined by applying a functional relationship defined for each weight to the metric to which it applies.

21. The method of claim 18, wherein the neighborhood has a dimension of 3 frames by 5 lines by 15 pixels per line.

22. A video filtering processor for reducing picture noise in a plurality of video frames configured to:

select a first pixel from one of the video frames, the first pixel having location coordinates at a horizontal position x, a vertical position y, and a time t;

determine weights for a set of pixel intensities in the video frames, wherein each weight is based on at least one pixel in a neighborhood of the first pixel and at least one pixel in a neighborhood of the pixel to be weighted;

determine a weighted average using the weights and the pixel intensities; and

identify the weighted average as the filtered output at pixel location x, y, and t.

23. The video filtering processor of claim 22, wherein the weight applied to a pixel is determined by a functional relationship to at least one of a group comprising the following metrics:

the spatial distance between the location coordinates x, y, and the location coordinates of the pixel being averaged;

the difference between the time t and a value of time associated with the pixel being averaged;

the spatial distance between the location coordinates x, y and the motion compensated values of the spatial location coordinates of the pixel being averaged;

the absolute value of the average difference between the intensity of the pixel at x, y, t and the intensity of the pixel being averaged;

the difference between the average intensity in a two-dimensional spatial neighborhood of the pixel being averaged and the average intensity in a spatial neighborhood of the pixel at the x and y; and

the sum of the absolute values of the difference between the intensity in a two-dimensional neighborhood of x and y and a point identically displaced from the pixel being averaged.

24. The video filtering processor of claim 22, wherein a group of weights is determined by applying a functional relationship defined for each weight to the metric to which it applies.

25. The method of claim 22, wherein the neighborhood has a dimension of 3 frames by 5 lines by 15 pixels per line.

26. A system for filtering, in three dimensions, pixels in a plurality of sequential video frames comprising:

means for selecting a first pixel from one of the video frames, the first pixel having location coordinates at a horizontal position x, a vertical position y, and a time t;

means for determining weights for a set of pixel intensities in the video frames, wherein each weight is based on at least one pixel in a neighborhood of the first pixel and at least one pixel in a neighborhood of the pixel to be weighted;

means for determining a weighted average using the weights and the pixel intensities; and

means for identifying the weighted average as the filtered output at pixel location x, y, and t.

27. The method of claim 26, wherein the neighborhood has a dimension of 3 frames by 5 lines by 15 pixels per line.