Directional video filters for locally adaptive spatial noise reduction

• US 20050135699A1
• Filed: 12/23/2003
• Published: 06/23/2005
• Est. Priority Date: 12/23/2003
• Status: Active Grant

• Current Assignee: Google Technology Holdings LLC (Alphabet Inc.)
• Original Assignee: General Instrument Corporation (CommScope Holding Company, Inc.)
• Inventors: Anderson, Candace M.
• Granted Patent: US 7,373,013 B2
• Time in Patent Office: Days
• US Class Current: 382/261
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20012 Locally adaptive
• CPC Class Codes: G06T 5/002 Denoising; Smoothing noise processing or correction adapted to be used in an image pickup device containing and electronic image sensor H04N23/81, H04N25/60 - H04N25/67
• CPC Class Codes: G06T 5/70 Denoising; Smoothing
• CPC Class Codes: H04N 5/213 Circuitry for suppressing or minimising impulsive noise for suppressing or minimising disturbance in image signal generation H04N23/81

Abstract

Spatial noise is reduced in an image having a plurality of pixels by detecting object boundaries and unstructured areas in the image and applying 3-tap high pass filters to each pixel in the image in at least four, but less than eight directions to determine the best direction for local low pass filtering. Low pass filtering is applied only along object boundaries and unstructured areas within the image so as to minimize the tendency to blur image edges. Using only four high pass filters to locate horizontal, vertical and diagonal image edges passing through the center of a 3×3 pixel array provides good results.

Claims

1. A method of reducing spatial noise in an image having a plurality of pixels, comprising:

detecting object boundaries and unstructured areas in the image by applying 3-tap high pass filters to each pixel in the image in at least four directions to determine the best direction(s) for local low pass filtering; and

applying low pass filtering only along object boundaries and unstructured areas within the image so as to minimize the tendency to blur image edges.

2. A method according to claim 1, further comprising:

when high-pass filtering results are nearly equal in absolute value with one another, applying multi-directional low pass filtering.

3. A method according to claim 1, further comprising:

when high-pass filtering results are nearly equal in absolute value, applying multi-directional low pass filtering without explicit tap weight changes.

4. A method according to claim 1, further comprising:

introducing an offset parameter (Δ

) to the high pass filters to promote smoothing.

5. A method according to claim 1, wherein the high pass filters have a minimum absolute value, and further comprising:

introducing an offset parameter (Δ

) to the high pass filters to promote smoothing even when the minimum absolute value of the high pass filters is zero.

6. A method according to claim 1, wherein:

the high pass filters have center coefficients; and

further comprising;

varying relative weighting of the center coefficients from zero to unity.

7. A method according to claim 1, wherein:

the low pass filtering is applied to pixel luminance values.

8. A method according to claim 1, further comprising:

restricting local low pass filtering to directions that do not cross image edges.

9. A method according to claim 1, wherein:

the high pass filters are 3-tap zero-mean high pass filters; and

further comprising;

determining an optimum direction for low pass filtering by using tap coefficients where the center tap coefficient is twice that of the taps applied to the neighboring pixels which have an opposite sign.

10. A method according to claim 1, further comprising:

using a set of only four high pass filters to locate horizontal, vertical and diagonal image edges passing through the center of a 3×

3 pixel array.

11. A method according to claim 1, further comprising:

choosing the direction for low pass filtering by comparing results of the high pass filters with a minimum absolute value for each of the selected ones of the pixels in the image.

12. A method according to claim 1, further comprising:

performing high pass filtering for the selected ones of the pixels in a horizontal, vertical and two diagonal directions.

13. A method according to claim 1, wherein said 3-tap high pass filters are applied to each pixel in the image in at least four, but less than eight directions to determine the best direction(s) for local low pass filtering.

14. A method of performing directional smoothing of an image, comprising:

for each pixel p(i,j) in a video frame of I rows by J columns, computing a set of zero-mean high-pass filters with tap weights, where the center tap coefficient is twice that of the taps applied to the neighboring pixels which have an opposite sign, for only the horizontal, vertical, and two diagonal directions; and

applying low pass filtering only along object boundaries and unstructured areas within the image so as to minimize the tendency to blur image edges.

15. A method according to claim 14, wherein each pixel is a center pixel in a 3×

3 array of pixels, the array having top, middle and bottom horizontal rows and leftmost, middle and rightmost vertical columns, further comprising;

leaving the top and bottom horizontal rows in the 3×

3 array unfiltered; and

leaving the leftmost and rightmost vertical columns unfiltered.

16. A method according to claim 14, further comprising:

determining the high-pass filter with the minimum absolute value as expressed in the following equation;

|hp_min(i,j,k_min)|=min_k(↑

hp_—

filt(i,j,k|)

17. A method according to claim 14, further comprising:

applying a unity gain 3×

3 low pass filter with tap weights, where the center tap coefficient is twice that of the taps applied to the neighboring pixels which have an opposite sign, in the direction of k_min for local smoothing of the image pixel p(i,j).

18. A method according to claim 17, wherein:

the low pass filter is calculated from the signed value of hp_filt(i,j,k_min) by simple subtraction, as in the following equation;

lp_—

filt(i,j)=p(i,j)−

(hp_—

filt(i,j,k_min)

19. A method according to claim 14, further comprising:

calculating distribution statistics for each filtered pixel by summing the number of high pass directions for which the absolute value matches that of the minimum.

20. A method according to claim 14, further comprising:

determining the number of equal directions, and

when the number of equal directions is greater than one, computing a local low pass filter by averaging over D directions of the high pass results, according to the following equation;

$\mathrm{lp\_filt}<br><br>\left(i,j\right)=p<br><br>\left(i,j\right)-\underset{m=1}{\overset{D}{\sum <br><br>}}<br><br>(\mathrm{hp\_filt}<br><br>\left(i,j,k_m\right)/D$

21. A method according to claim 20, further comprising:

creating a variety of filter shapes without making explicit changes to coefficient values.

22. A method according to claim 20, further comprising:

introducing a non-negative offset parameter (Δ

) so that all directions of k for which |hp_filt(i,j,k)|−

hp_min(i,j,k_min)≦

Δ

are included in the low-pass average.

23. A method according to claim 22, wherein:

a value of the offset parameter (Δ

) greater than zero promotes low pass filtering at p(i,j) where hp_min(i,j,k_min)=0, thereby decreasing the number of pixels for which no local smoothing would otherwise be applied.

24. A method of performing spatial noise filtering of an image, comprising:

applying a plurality of 3-tap zero-mean high pass filters to determine a plurality of low pass filter shapes,

varying the relative weight of the center tap of the high pass filters in the range of unity down to zero.

25. A method according to claim 24, further comprising:

automatically varying the center tap weighting variable is to any value between two parameters depending upon the local value of hp_min(i,j,k_min)+Δ

.