Noise reduction in a digital video

• US 20060114994A1
• Filed: 12/01/2004
• Published: 06/01/2006
• Est. Priority Date: 12/01/2004
• Status: Abandoned Application

• Current Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Silverstein, D. Amnon
• Application Number:
  US11/001,101
• Publication Number:
  US 20060114994A1
• Time in Patent Office: Days
• US Class Current: 375/240.120
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20182 Noise reduction or smoothing in the temporal domain; Spatio-temporal filtering
• CPC Class Codes: G06T 2207/20204 Removing film grain; Adding simulated film grain
• CPC Class Codes: G06T 5/002 Denoising; Smoothing noise processing or correction adapted to be used in an image pickup device containing and electronic image sensor H04N23/81, H04N25/60 - H04N25/67
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 5/70 Denoising; Smoothing
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: H04N 5/147 Scene change detection
• CPC Class Codes: H04N 5/21 Circuitry for suppressing or minimising disturbance, e.g. moiré or halo

Abstract

A method performed by a processing system is provided. The method comprises determining a correspondence between a first pixel value in a first frame of a digital video and a second pixel value in a second frame of the digital video, processing the first pixel value and the second pixel value to remove noise and generate a third pixel value, and re-mapping the first pixel value in the first frame with the third pixel value.

Claims

1. A method performed by a processing system, the method comprising:

determining a correspondence between a first pixel value in a first frame of a digital video and a second pixel value in a second frame of the digital video;

processing the first pixel value and the second pixel value to remove noise and generate a third pixel value; and

re-mapping the first pixel value in the first frame with the third pixel value.

2. The method of claim 1 further comprising:

performing motion compensation on the second frame relative to the first frame prior to determining the correspondence.

3. The method of claim 1 further comprising:

determining a correspondence between the first pixel value in the first frame and a fourth pixel value in a third frame of the digital video; and

processing the first pixel value, the second pixel value, and the fourth pixel value to remove noise and generate the third pixel value.

4. The method of claim 3 wherein the second frame is prior to the first frame in the digital video, and wherein the third frame is subsequent to the first frame in the digital video.

5. The method of claim 1 further comprising:

identifying a temporal window that comprises the first pixel value and the second pixel value prior to processing the first pixel value and the second pixel value.

6. The method of claim 1 further comprising:

identifying a temporal window that comprises the first pixel value and the second pixel value in response to determining that the second pixel value is within a threshold prior to processing the first pixel value and the second pixel value.

7. The method of claim 1 further comprising:

adding the second pixel value to an output buffer value corresponding to a pixel associated with the first pixel value in an output buffer in response to determining that a difference between the second pixel value and the first pixel value is less than a threshold;

incrementing a denominator buffer value corresponding to the pixel associated with the first pixel value in a denominator buffer in response to determining that the difference is less than the threshold; and

dividing the output buffer value by the denominator buffer value to generate the third pixel value.

8. The method of claim 7 further comprising:

setting a mask value in a temporal mask to indicate a false condition in response to detecting that a difference between a fourth pixel value in a third frame of the digital video and the first pixel value is greater than the threshold.

9. The method of claim 1 further comprising:

processing the first pixel value and the second pixel value to remove noise and generate the third pixel value according to a confidence measurement associated with the correspondence.

10. The method of claim 9 wherein the third pixel value comprises a weighted average of the first pixel value and the second pixel value.

11. The method of claim 9 further comprising:

computing the confidence measurement associated with the correspondence.

12. The method of claim 1 further comprising:

re-mapping the second frame relative to the first frame using a set of motion vectors computed between the second frame and a third frame in response to not detecting a scene change associated with the first frame.

13. A system comprising:

a processor; and

a memory system for storing a digital video and a video enhancement module;

wherein the processor is configured to execute the video enhancement module to;

determine a correspondence between a first plurality of pixel values in a first frame of a digital video and a second plurality of pixel values in a second frame of the digital video;

process the first plurality of pixel values and the second plurality of pixel values to remove noise and generate a third plurality of pixel values; and

re-map the first plurality of pixel values in the first frame with the third plurality of pixel values.

14. The system of claim 13 wherein the processor is configured to execute the video enhancement module to:

perform motion compensation on the second frame relative to the first frame prior to determining the correspondence.

15. The system of claim 13 wherein the processor is configured to execute the video enhancement module to:

determine a correspondence between the first plurality of pixel values in the first frame and a fourth plurality of pixel values in a third frame of the digital video; and

process the first plurality of pixel values, the second plurality of pixel values, and the fourth plurality of pixel values to remove noise and generate the third plurality of pixel values.

16. The system of claim 13 wherein the processor is configured to execute the video enhancement module to:

identify a temporal window for each of the first plurality of pixel values by identifying the second plurality of pixel values that are within a threshold prior to processing the first plurality of pixel values and the second plurality of pixel values.

17. The system of claim 13 wherein the processor is configured to execute the video enhancement module to:

add each of the second plurality of pixel values whose difference between a corresponding one of the first plurality of pixel values is less than a threshold to a corresponding one of a plurality of output buffer values in an output buffer;

increment a corresponding one of a plurality of denominator buffer values in a denominator buffer for each of the second plurality of pixel values whose difference between a corresponding one of the first plurality of pixel values is less than the threshold; and

divide the plurality of output buffer values by the plurality of denominator buffer values to generate the third plurality of pixel values.

18. The system of claim 17 wherein the processor is configured to execute the video enhancement module to:

set a corresponding one of a plurality of mask values in a temporal mask to indicate a false condition for each of the second plurality of pixel values whose difference between a corresponding one of the first plurality of pixel values is greater than the threshold.

19. The system of claim 13 wherein the processor is configured to execute the video enhancement module to:

process the first pixel value and the second pixel value to remove noise and generate the third pixel value according to a confidence measurement associated with the correspondence.

20. The system of claim 19 wherein the third pixel value comprises a weighted average of the first pixel value and the second pixel value.

21. The system of claim 19 wherein the processor is configured to execute the video enhancement module to:

compute the confidence measurement associated with the correspondence.

22. The system of claim 13 wherein the processor is configured to execute the video enhancement module to:

re-map the second frame relative to the first frame using a set of motion vectors computed between the second frame and a third frame in response to not detecting a scene change associated with the first frame.

23. A system comprising:

means for determining a first correspondence between a first pixel value in a first frame of a digital video and a second pixel value in a second frame of the digital video;

means for determining a second correspondence between the first pixel value in the first frame and a third pixel value in a third frame of the digital video;

means for processing the first pixel value, the second pixel value, and the third pixel value to remove noise and generate a fourth pixel value; and

means for re-mapping the first pixel value in the first frame with the fourth pixel value.

24. The system of claim 23 further comprising:

means for performing motion compensation on the second frame relative to the first frame prior to determining the first correspondence; and

means for performing motion compensation on the third frame relative to the first frame prior to determining the second correspondence.

25. The system of claim 23 wherein the second frame is prior to the first frame in the digital video, and wherein the third frame is subsequent to the first frame in the digital video.

26. The system of claim 23 further comprising:

means for identifying a temporal window that comprises the first pixel value, the second pixel value, and the third pixel value prior to processing the first pixel value, the second pixel value, and the third pixel value.

27. The system of claim 23 further comprising:

means for identifying a temporal window that comprises the first pixel value, the second pixel value, and the third pixel value in response to determining that the second pixel value and the third pixel value are within a threshold prior to processing the first pixel value, the second pixel value, and the third pixel value.

28. The system of claim 23 further comprising:

means for adding the second pixel value to an output buffer value corresponding to the first pixel value in an output buffer in response to determining that a first difference between the second pixel value and the first pixel value is less than a threshold;

means for adding the third pixel value to the output buffer value corresponding to the first pixel value in the output buffer in response to determining that a second difference between the third pixel value and the first pixel value is less than the threshold;

means for incrementing a denominator buffer value corresponding to the first pixel value in a denominator buffer in response to determining that the first difference is less than the threshold; and

means for incrementing the denominator buffer value corresponding to the first pixel value in the denominator buffer in response to determining that the second difference is less than the threshold;

means for dividing the output buffer value by the denominator buffer value to generate the fourth pixel value.

29. The system of claim 28 further comprising:

means for setting a mask value corresponding to the first pixel in a temporal mask to indicate a false condition in response to detecting that a difference between a fourth pixel value in a fourth frame of the digital video and the first pixel value is greater than the threshold.

30. The system of claim 23 further comprising:

processing the first pixel value and the second pixel value to remove noise and generate the third pixel value according to a confidence measurement associated with the correspondence.

31. The system of claim 30 wherein the third pixel value comprises a weighted average of the first pixel value and the second pixel value.

32. The system of claim 30 further comprising:

computing the confidence measurement associated with the correspondence.

33. The system of claim 23 further comprising:

re-mapping the second frame relative to the first frame using a set of motion vectors computed between the second frame and a third frame in response to not detecting a scene change associated with the first frame.

34. A program product comprising a computer-readable medium including instructions executable by a processing system for:

determining a correspondence between a first pixel value in a first frame of a digital video and a second pixel value in a second frame of the digital video;

processing the first pixel value and the second pixel value to remove noise and generate a third pixel value; and

re-mapping the first pixel value in the first frame with the third pixel value.

35. The program product of claim 34 wherein the computer-readable medium includes instructions executable by the processing system for:

performing motion compensation on the second frame relative to the first frame prior to determining the correspondence.

36. The program product of claim 34 wherein the computer-readable medium includes instructions executable by the processing system for:

identifying a temporal window that comprises the first pixel value and the second pixel value prior to processing the first pixel value and the second pixel value.

37. The program product of claim 34 wherein the computer-readable medium includes instructions executable by the processing system for:

processing the first pixel value and the second pixel value to remove noise and generate the third pixel value according to a confidence measurement associated with the correspondence.

38. The program product of claim 34 wherein the computer-readable medium includes instructions executable by the processing system for:

re-mapping the second frame relative to the first frame using a set of motion vectors computed between the second frame and a third frame in response to not detecting a scene change associated with the first frame.