System and method for video noise reduction using an adaptive temporal method with motion detection and motion compensation

• US 20080204600A1
• Filed: 02/23/2007
• Published: 08/28/2008
• Est. Priority Date: 02/23/2007
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Kim, Yeong-Taeg, Xu, Ning
• Granted Patent: US 8,059,207 B2
• Time in Patent Office: Days
• US Class Current: 348/607
• CPC Class Codes: H04N 5/144 Movement detection for video coding H04N19/503; analysis of motion in general G06T7/20
• CPC Class Codes: H04N 5/21 Circuitry for suppressing or minimising disturbance, e.g. moiré or halo

Abstract

An adaptive temporal noise reduction method that adaptively combines motion adaptive filtering results and motion compensated results to reduce Gaussian additive noise in video sequences is described herein. The system determines the motion detection and motion compensation results from the current frame and the filtered previous frame. Measurements on the video are used to determine a probabilistic measure of noise that is employed to adaptively combine the motion detection and motion compensation results.

Claims

1. A method for temporally filtering noise from a plurality of sequential video frames comprising:

selecting a first pixel from a current frame at a location (x,y) having an intensity I_t(x,y);

selecting a second pixel at (x,y) in a filtered frame immediately preceding the current frame, the filtered second pixel having an intensity denoted by Î

_t-1(x,y);

determining a motion vector [v_x,v_y] that provides a close match between a neighborhood containing the first pixel and a similarly configured neighborhood in the filtered frame that immediately precedes the first frame in time;

determining the intensity of a third pixel at a location that is offset from the first pixel'"'"'s by the magnitude and direction of the motion vector, the location being (x−

v_x,y−

v_y) in the filtered frame immediately preceding the current frame, and the intensity being denoted by Î

_t-1(x−

v_x,y−

v_y); and

computing the output of the temporal filter at the location (x,y) to be a weight averaged sum of three intensity values;

the intensities of the first pixel, the second pixel and the third pixel.

2. The method of claim 1, wherein a metric md is determined that is indicative of relative motion between a neighborhood of the first pixel and an identically configured neighborhood of the second pixel.

3. The method of claim 1, wherein a metric mc is determined that is indicative of the measure of fit between a neighborhood of the first pixel and a similarly configured motion compensated neighborhood in the filtered frame that immediately precedes the current frame.

4. The method of claim 1, further comprising two components p₁ and p₂, of a quantity p_noise, wherein p₁ is found by counting the peaks and valleys of the intensity in a local neighborhood of the first selected pixel in the current frame and soft-switching the result, and p₂ is found by computing the local standard deviation of the pixels'"'"' intensity and soft switching the result;

and wherein p_noise is equal to p₁ multiplied by p₂.

5. The method of claim 1, wherein the weights applied to I_t(x,y), I_t-1(x,y) and Î

_t-1(x−

v_x,y−

v_y) are based on md, mc, and p_noise.

6. The method of claim 1, further comprising:

evaluating a limit L based on the probability associated with the noise;

selecting the filter'"'"'s output to be Î

_t′

unless |I_t−

Î

_t′

| is greater than L; and

selecting the filter'"'"'s output to be either I_t+L or I_t−

L, depending on which value is closer to Î

_t′

.

7. The method of claim 1, wherein the computation of the weight averaged sum comprises:

combining I_t(x,y) and Î

_t-1(x,y) to obtain Î

_md(x,y) using weights derived from md;

combining I_t(x,y) and I_t-1(x−

v_x, y−

v_y) to obtain Î

_mc using weights derived from mc;

combining Î

_mc(x,y) and Î

_md(x,y) to get Î

_mdmc(x,y) using a weight derived from mc, md and p_noise;

combining I_t(x,y) and Î

_mdmc(x,y) to get I_t(x,y) using a weight derived from the three measured variables mc, md and p_noise; and

selecting either I_t′

(x,y) or an amplitude limited version of I_t(x,y), depending on a limit value derived from p_noise.

8. The method of claim 7, wherein the weight used to form the weighted average of the first pixel from the current frame and the second pixel from the filtered second frame is found by passing md through a soft switcher to form ŵ

_md.

9. The method of claim 8, wherein the weight used to form the weighted average of the first pixel from the current frame and the third pixel from the filtered second frame that is motion compensated is found by passing mc through a soft switcher to form ŵ

_mc.

10. The method of claim 9, wherein averaging the pixel intensities Î

_md(x,y) and Î

_mc(x,y) comprises finding three components to a weight α

as follows;

a weight α

₁ that is found by passing p_noise through a soft-switcher;

a component α

₂ that is found by putting md in a soft-switcher;

a component α

₃ that is found by passing mc through a soft-switcher; and

wherein α

is defined as a quantity (1−

α

₁·

α

₂)α

₃.

11. The method of claim 10, wherein the weight used to average Î

^md(x,y) and Î

_mc(x,y) is α

.

12. The method of claim 11, further comprising a ratio r_md and the weight ŵ

_md, wherein r_md is found by soft-switching md, ŵ

_md is found by soft switching md, and the soft switching that forms ŵ

_md is based on a weight determined in forming the filtered frame that immediately precedes the current frame.

13. The method of claim 12, further comprising a ratio r_mc and a weight ŵ

_mc, wherein r_mc is found by soft switching mc, ŵ

_mc is found by soft switching mc, and the soft switching that forms ŵ

_mc is based on a weight found in forming the filtered frame that immediately precedes the current frame.

14. The method of claim 13, further comprising a weight r_t that is found by weight averaging r_mc and r_md using α

as the weight.

15. The method of claim 14, wherein a quantity ŵ

_t that is computed as the weighted average of ŵ

_md and ŵ

_mc using α

as the weight that is further limited to a maximum value to form a final weight ŵ

_t that is associated with coordinates x, y and t.

16. The method of claim 15, wherein ŵ

_t will be used in filtering the pixel at (x,y,t+1).

17. The method of claim 16, wherein a preliminary version of the filter'"'"'s output Î

_t′

is formed as the weighted average of Î

_mdmc and I_t based on the weights r_t and (1−

r_t).

18. A system for filtering noise from a plurality of video frames, comprising

an adaptive temporal noise reduction block that is configured to utilize motion detection and motion compensation metrics based on a current frame and a filtered frame that immediately precedes it to form filtered pixels that comprise a current filtered frame.

19. The system of claim 18, further comprising:

a buffer configured to provide a selected first pixel'"'"'s video intensity from the current frame;

a first frame buffer configured to provide at least one pixel'"'"'s video intensity from the filtered frame that immediately precedes the current frame; and

a second frame buffer configured to store a weight computed in filtering the frame that immediately precedes the current frame and to provide it in filtering the current frame.

20. The system of claim 18, wherein the adaptive temporal noise reduction block comprises:

a motion detection block configured to detect relative motion between an area containing the first pixel in the current frame and an identically configured area in the filtered frame that immediately precedes the current frame and to generate a metric md that is indicative of that motion;

a motion estimation block configured to find a motion vector mv that is indicative of a relative displacement between an area containing the first pixel in the present frame and a similarly configured area in the filtered frame that immediately precedes it;

a motion compensation block configured to compute a metric mc indicative of the measure of fit between an area containing the first pixel in the current frame and a similarly configured motion compensated area in the filtered frame that immediately precedes the current frame; and

a noise adaptive filtering block that is configured to accept md, mv and mc as inputs;

wherein the noise adaptive filtering block is configured to accept measurements of the variance of the noise in a neighborhood of the selected pixel, inputs from the three buffers, and to compute values of the weight that are stored in the third buffer and read out these values to form filtered pixels that comprise the current filtered frame.

21. The system of claim 19, wherein the inputs processed by noise adaptive filtering block further comprise:

the value of the first pixel'"'"'s intensity from the buffer;

two pixel intensities from the second buffer;

the value of the weight from the third buffer; and

the value of the noise variance in a neighborhood of the first pixel.