Spatial and temporal filtering mechanism for digital motion video signals

• US 6,281,942 B1
• Filed: 08/11/1997
• Issued: 08/28/2001
• Est. Priority Date: 08/11/1997
• Status: Expired due to Term

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Wang, Albert S.
• Primary Examiner(s): Eisenzopf, Reinhard J.
• Assistant Examiner(s): Lo, Linus H.
• Application Number:
  US08/909,202
• Time in Patent Office: 1,478 Days
• Field of Search: H04/N.521, H04/N.5213, H04/N.5217, 348/606, 348/607, 348/608, 348/620, 348/624, 348/431.1, 348/451, 348/452, 348/610, 382/260, 382/262, 382/264
• US Class Current: 348/607
• CPC Class Codes: H04N 19/117 Filters, e.g. for pre-processing or post-processing sub-band filter banks H04N19/635
• CPC Class Codes: H04N 19/124 Quantisation
• CPC Class Codes: H04N 19/132 Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
• CPC Class Codes: H04N 19/14 Coding unit complexity, e.g. amount of activity or edge presence estimation H04N19/146 takes precedence
• CPC Class Codes: H04N 19/176 the region being a block, e.g. a macroblock
• CPC Class Codes: H04N 19/182 the unit being a pixel
• CPC Class Codes: H04N 19/587 involving temporal sub-sampling or interpolation, e.g. decimation or subsequent interpolation of pictures in a video sequence
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 19/80 Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation H04N19/635, H04N19/86 take precedence
• CPC Class Codes: H04N 21/234354 by altering signal-to-noise ratio parameters, e.g. requantization
• CPC Class Codes: H04N 21/234381 by altering the temporal resolution, e.g. decreasing the frame rate by frame skipping
• CPC Class Codes: H04N 21/426 Internal components of the client ; Characteristics thereof H04N5/44 takes precedence
• CPC Class Codes: H04N 5/21 Circuitry for suppressing or minimising disturbance, e.g. moiré or halo
• CPC Class Codes: H04N 5/213 Circuitry for suppressing or minimising impulsive noise for suppressing or minimising disturbance in image signal generation H04N23/81

Abstract

Frames of a digital video signal are spatially filtered to remove impulse and other noise from the video signal. The spatially filtered frame is temporally filtered to further remove noise from the digital video signal. The spatial filter is adaptive, heavily filtering portions of low detail in the represented subject matter while lightly filtering or not filtering portions of moderate to high detail in the represented subject matter. As a result, clarity and detail in the subject matter of the digital video signal are preserved. The temporal filter is similarly adaptive and determines if and to what degree the subject matter of portions of the spatially filtered frame correlate with corresponding portions of a previous frame. By previously spatially filtering the frame, determinations regarding the correlation of the current frame with corresponding portions of the previous, which is also spatially and temporally filtered, are more accurate. Accordingly, temporal filtering can be performed more aggressively without temporally blurring the digital video signal. The result of spatial and temporal filtering of the digital video signal is significant reduction in noise without degradation of the signal and improved compression of the digital video signal.

Claims

1. A method for filtering at least a portion of a digital video signal, the method comprising:

calculating a pixel variance for a block of source pixels from a current frame of the motion video image;

applying a heavy spatial filter to the block of source pixels to form a block of spatially filtered pixels representing current subject matter of the current frame when the pixel variance is less than a first pixel variance threshold;

applying a light spatial filter to the block of source pixels to form the block of spatially filtered pixels when the pixel variance is less than a second pixel variance threshold;

calculating a pixel error between the block of spatially filtered pixels and a block of pixels of a previous frame of the digital video signal;

applying a heavy temporal filter to the block of spatially filtered pixels when the pixel error is less than a first pixel error threshold; and

applying a light temporal filter to the block of spatially filtered pixels when the pixel error is less than a second pixel error threshold.

2. The method of claim 1 wherein the step of applying a light spatial filter comprises

applying a five pixel median filter and a five pixel smoothing filter to the block of source pixels, and further wherein the step of applying a heavy spatial filter comprises applying a five pixel median filter and a nine pixel smoothing filter to the block of source pixels.

3. The method of claim 1 wherein the calculated pixel error is an absolute pixel difference.

11. The method of claim 1, wherein the step of applying a heavy temporal filter includes applying a two-tap infinite impulse response filter in which the pixels of the previous frame are weighted by a factor of substantially equal to 0.75, and further wherein the step of applying a light temporal filter includes applying a two-tap infinite impulse response filter in which the pixels of the previous frame are weighted by a factor of substantially equal to 0.5.

4. A computer readable medium useful in association with a computer which includes

a processor and a memory, the computer readable medium including computer instructions which are configured to cause the computer to filter a digital video signal by performing the steps of:

calculating a pixel variance for a block of source pixels from a current frame of the motion video image;

applying a heavy spatial filter to the block of source pixels to form a block of spatially filtered pixels representing current subject matter of the current frame when the pixel variance is less than a first pixel variance threshold;

applying a light spatial filter to the block of source pixels to form the block of spatially filtered pixels when the pixel variance is less than a second pixel variance threshold;

calculating a pixel error between the block of spatially filtered pixels and a block of pixels of a previous frame of the digital video signal;

applying a heavy temporal filter to the block of spatially filtered pixels when the pixel error is less than a first pixel error threshold; and

applying a light temporal filter to the block of spatially filtered pixels when the pixel error is less than a second pixel error threshold.

5. The computer readable medium of claim 4 wherein the step of applying a light spatial filter comprises

applying a five pixel median filter and a five pixel smoothing filter to the block of source pixels, and further wherein the step of applying a heavy spatial filter comprises applying a five pixel median filter and a nine pixel smoothing filter to the block of source pixels.

6. The computer readable medium of claim 4 wherein the calculated pixel error is an absolute pixel difference.

12. The computer readable medium of claim 4, wherein the step of applying a heavy temporal filter includes applying a two-tap infinite impulse response filter in which the pixels of the previous frame are weighted by a factor of substantially equal to 0.75, and further wherein the step of applying a light temporal filter includes applying a two-tap infinite impulse response filter in which the pixels of the previous frame are weighted by a factor of substantially equal to 0.5.

7. A computer system comprising:

a processor,

a memory operatively coupled to the processor; and

a digital video signal filter which executes in the processor from the memory and which, when executed by the processor, causes the computer to filter a digital video signal by performing the steps of;

calculating a pixel variance for a block of source pixels from a current frame of the motion video image;

applying a heavy spatial filter to the block of source pixels to form a block of spatially filtered pixels representing current subject matter of the current frame when the pixel variance is less than a first pixel variance threshold;

applying a light spatial filter to the block of source of pixels to form the block of spatially filtered pixels when the pixel variance is less than a second pixel variance threshold;

calculating a pixel error between the block of spatially filtered pixels and a block of pixels of a previous frame of the digital video signal;

applying a heavy temporal filter to the block of spatially filtered pixels when the pixel error is less than a first pixel error threshold; and

applying a light temporal filter to the block of spatially filtered pixels when the pixel error is less than a second pixel error threshold.

8. The computer system of claim 7 wherein the step of applying a light spatial filter comprises

applying a five pixel median filter and a five pixel smoothing filter to the block of source pixels, and further wherein the step of applying a heavy spatial filter comprises applying a five pixel median filter and a nine pixel smoothing filter to the block of source pixels.

9. The computer system of claim 7 wherein the calculated pixel error is an absolute pixel difference.

10. The computer system of claim 7 wherein the step of comparing comprises:

measuring a difference between the block of spatially filtered pixels and the block of pixels of the previous frame; and

determining whether the block of pixels of the previous frame represents the current subject matter by comparison of the measured difference to a predetermined threshold.

13. The computer system of claim 7, wherein the step of applying a heavy temporal filter includes applying a two-tap infinite impulse response filter in which the pixels of the previous frame are weighted by a factor of substantially equal to 0.75, and further wherein the step of applying a light temporal filter includes applying a two-tap infinite impulse response filter in which the pixels of the previous frame are weighted by a factor of substantially equal to 0.5.