Directional spatial video noise reduction

• US 7,437,013 B2
• Filed: 03/24/2004
• Issued: 10/14/2008
• Est. Priority Date: 12/23/2003
• Status: Expired due to Fees

• Current Assignee: Google Technology Holdings LLC (Alphabet Inc.)
• Original Assignee: General Instrument Corporation (CommScope Holding Company, Inc.)
• Inventors: Anderson, Candace M.
• Primary Examiner(s): Kassa; Yosef
• Application Number:
  US10/809,022
• Publication Number:
  US 20050135700A1
• Time in Patent Office: 1,665 Days
• Field of Search: 382/260, 382/261, 382/263, 382/264, 382/275, 382/274, 382/305, 358/3.26, 358/3.27, 358/463
• US Class Current: 382/261
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20012 Locally adaptive
• CPC Class Codes: G06T 5/70 Denoising; Smoothing
• CPC Class Codes: H04N 5/213 Circuitry for suppressing or minimising impulsive noise for suppressing or minimising disturbance in image signal generation H04N23/81

Abstract

A method of reducing spatial noise in an image. Low-pass (smoothing) filters are calculated simultaneously from three successive image rows. Three blocks (m1, m2, m3) are associated with the three successive image rows, and the blocks are processed in row-major order. This implementation is applicable to both luminance and chrominance. The number of smoothing parameters is reduced to one. The technique is applicable to both luminance and chrominance. Directional mapping is used. Extension of the technique to spatial filtering using a 5×5 neighborhood (using five successive image rows) is described. Embodiments of the method using the MMX instruction set are described.

Claims

1. A method of reducing spatial noise in an image having a plurality of pixels, the pixels being arranged in a plurality of (h) rows and (w) columns, comprising:

providing a set of three 64 bit registers, each register representing eight horizontally adjacent pixel values from one of three respective vertically adjacent image rows;

computing eight sets of directional high-pass values, one for each horizontal pixel position represented by said registers; and

computing directionally smoothed low pass pixel values by combining said high-pass values with image pixel values to produce directionally weighted sums; and

using the weighted sums to provide a reduced spatial noise image.

2. A method according to claim 1, wherein:

said three 64-bit registers include a first (up) register, a second (mid) register and a third (down) register, said first, second and third registers each representing pixel values for eight horizontally adjacent image pixels;

pixel values in said first (up) register represent image pixels horizontally aligned with and vertically adjacent to pixels represented by corresponding pixel values in said second (mid) register; and

pixel values in said second (mid) register represent image pixels horizontally aligned with and vertically adjacent to pixels represented by corresponding pixel values in said third (down) register.

3. A method according to claim 2, wherein:

said step of computing eight sets of directional high-pass values further comprises;

computing eight average values for corresponding first and third image row pixels in said first (up) and third (down) registers;

computing eight maxima and eight minima between said average values and corresponding second image row pixels in said second (mid) register;

determining eight high-pass absolute values by subtracting said minima from said maxima;

setting values of eight sign bytes such that if corresponding second row pixel values and minima are equal, a corresponding sign byte is set to all 1'"'"'s, otherwise the corresponding sign byte is set to all 0'"'"'s;

shifting the first (up) and third (down) registers for left diagonal alignment of pixel values;

calculating left diagonal high-pass absolute values and signs;

shifting the first (up) and third (down) registers for right diagonal alignment of pixel values;

calculating right diagonal high-pass absolute values and signs;

shifting said second (mid) register for horizontal alignment of pixel values; and

calculating horizontal high-pass absolute values and signs.

4. A method according to claim 2, wherein:

said step of computing directionally smoothed low pass pixel values further comprises;

clearing (zeroing) high-pass values to exclude from low pass filters;

clearing (zeroing) sign bytes for high pass values equal to zero;

computing low pass values for four upper bytes by summing four corresponding high-pass values and subtracting from four corresponding second image row pixel values in said second (mid) register; and

computing low pass values for four lower bytes by summing four corresponding high-pass values and subtracting from four corresponding second image row pixel values in said second (mid) register.

5. A method according to claim 2, wherein:

the first (up) register represents pixels p(i−

1, j+k) in a corresponding first (up) image row;

the second (mid) register represents pixels p(i, j+k) in a corresponding second (mid) image row; and

the third (down) register represents pixels p(i+1, j+k) in a corresponding third (down) image row.

6. A method according to claim 5, further comprising:

storing directional smoothing results for pixels p(i, j+k) in place of p(i−

1, j+k).

7. A method according to claim 5, further comprising:

computing scaled high-pass filters (hp_(i,j+,k,d)) for an aligned vertical direction (d=1) for eight horizontally contiguous pixels simultaneously using the relationship;

hp_filt(i,j+,k,d)=p(i,j+k)−

(p(i−

1,j+k)+p(i+1,j+k))/2 k=[0,7].

8. A method, according to claim 5, further comprising:

calculating low-pass filter values (lp_MMX(i,j+k) ) in accordance with the relationship;

$\mathrm{lp\_MMX}<br><br><br><br>\left(i,j+k\right)=\left[4·<br><br>p<br><br>\left(i,j+k\right)-\underset{d-1}{\overset{D}{\sum <br><br>}}<br><br><br><br>\mathrm{hp\_filt}<br><br><br><br>\left(i,j+k,d\right)\right]⪢<br><br>2<br><br>$ $k=\left[0,7\right].$

9. A method according to claim 5, further comprising:

determining a degree of directional smoothing for each pixel according to a number of directional high-pass values (hp_filt(i,j+k,d) ) that have high-pass absolute values less than or equal to |hp_min(i,j+k)+Δ

|.

10. A method of reducing spatial noise in an image having a plurality of pixels, the pixels being arranged in a plurality of (h) rows and (w) columns, comprising:

providing a set of 64 bit registers for representing eight 8-bit pixel values, said registers including a first (up) register for representing eight horizontally adjacent pixel values from a first image row;

a second (mid) register for representing eight horizontally adjacent pixel values from a second image row, said second image row pixels occurring vertically adjacent to and horizontally aligned with said first row pixels; and

a third (down) register for representing eight horizontally adjacent pixel values from a third image row, said third image row pixels occurring vertically adjacent to and horizontally aligned with said second image row pixels;

loading said first (up), second (mid) and third (down) registers with pixel values from three vertically adjacent image row and saving previous register contents;

computing eight average values for corresponding first and third image row pixels in said first (up) and third (down) registers;

computing eight maxima and eight minima between said average values and corresponding second image row pixels in said second (mid) register;

determining eight high-pass absolute values by subtracting said minima from said maxima;

setting values of eight sign bytes such that if corresponding second row pixel values and minima are equal, a corresponding sign byte is set to all 1'"'"'s, otherwise the corresponding sign byte is set to all 0'"'"'s;

shifting the first (up) and third (down) registers for left diagonal alignment of pixel values;

calculating left diagonal high-pass absolute values and signs;

shifting the first (up) and third (down) registers for right diagonal alignment of pixel values;

calculating right diagonal high-pass absolute values and signs;

shifting said second (mid) register for horizontal alignment of pixel values;

calculating horizontal high-pass absolute values and signs;

clearing (zeroing) high-pass values to exclude from low pass filters;

clearing (zeroing) sign bytes for high pass values equal to zero;

computing low pass values for four upper bytes by summing four corresponding high-pass values and subtracting from four corresponding second image row pixel values in said second (mid) register;

computing low pass values for four lower bytes by summing four corresponding high-pass values and subtracting from four corresponding second image row pixel values in said second (mid) register; and

packing and storing said low pass values into said first register.

11. A method of reducing spatial noise in an image having a plurality of pixels, the pixels being arranged in a plurality of (h) rows and (w) columns, comprising:

calculating a plurality of high-pass filter values for a number of pixels based upon surrounding pixel values in a 5×

5 pixel area surrounding each pixel for which a high-pass filter value is to be calculated;

determining directionality information for each from the high-pass filter values for each pixel;

determining directional low-pass (smoothing) filter values for each pixel based upon the directionality information for each pixel;

applying the low-pass filter values (smoothing) to each pixel according to pixel values in the 5×

5 pixel area surrounding each pixel; and

displaying the spatially reduced noise image.

12. A method according to claim 11, wherein each high-pass filter calculation comprises a summation of products, each product being determined by multiplying each pixel value in a 5×

5 pixel area surrounding each pixel by a corresponding high-pass filter coefficients.

13. A method according to claim 12, wherein directionality is determined by identifying a high-pass filter value with a highest absolute value above a threshold value.

14. A method according to claim 13, wherein the threshold value is determined by adding a constant to the value of the high-pass filter having a minimum absolute value.

15. A method according to claim 11, wherein the directional low pass filter values are determined to decrease the relative amount of smoothing applied in a direction indicated by the directionality information.

16. A method according to claim 11, wherein the number of low pass (smoothing) filters for each pixel value is equal to the number of corresponding high pass filters.

17. A method according to claim 16, wherein the number of high pass filters for each pixel value is four.

18. A method according to claim 17, wherein the filter directions are horizontal, vertical, diagonal right and diagonal left.

19. A method according to claim 16, wherein the number of high pass filters for each pixel value is eight.

20. A method according to claim 19, wherein the filter directions are, horizontal, vertical, diagonal right, diagonal left, greater diagonal right, greater diagonal left, lesser diagonal right and lesser diagonal left.