System for manipulating noise in digital images

• US 6,990,252 B2
• Filed: 05/23/2001
• Issued: 01/24/2006
• Est. Priority Date: 05/23/2000
• Status: Expired due to Term

• Current Assignee: Adobe Inc.
• Original Assignee: Adobe Systems Incorporated (Adobe Inc.)
• Inventors: Shekter, Jonathan Martin
• Primary Examiner(s): Johns, Andrew W.
• Assistant Examiner(s): Edwards, Patrick L.
• Application Number:
  US09/863,025
• Publication Number:
  US 20020034337A1
• Time in Patent Office: 1,707 Days
• Field of Search: 382/109, 382159-160, 382/164, 382/171, 382/207, 382272-273, 358/453, 358/463, 702189-195, 702/199
• US Class Current: 382/276
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/20016 Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform
• CPC Class Codes: G06T 2207/20076 Probabilistic image processing
• CPC Class Codes: G06T 2207/20204 Removing film grain; Adding simulated film grain
• CPC Class Codes: G06T 5/10 using non-spatial domain filtering
• CPC Class Codes: G06T 5/20 using local operators
• CPC Class Codes: G06T 5/70 Denoising; Smoothing
• CPC Class Codes: G06V 10/30 Noise filtering

Abstract

An apparatus for analyzing the broadband noise content of a digital image comprising the following: a means of automatically identifying regions of originally constant color in the image by analysis of the variance of pixel values of regions of the image; a means of automatically detecting and discarding regions deemed to be unrepresentative of the true noise content of an image, including under- and over-exposed regions; a means of allowing the user to manually select some or all required constant color regions if desired; and, a means of analyzing such constant color regions to generate a parametric or non-parametric model of the noise in the image, including frequency characteristic within and between channels and other characteristics such as phase which might describe structured noise.

Claims

1. A method of analyzing noise content in a digital image, the method comprising:

identifying regions of approximately constant pixel value within the digital image;

selecting a predetermined number of said identified regions based on pixel value variances within each said identified region;

analyzing said selected regions to generate a mathematical model of noise present in the digital image, wherein said analyzing said selected regions to generate a mathematical model of the noise present in the digital image comprises;

processing each selected region to remove a mean pixel value from each pixel in the selected region and obtain an approximation of the noise in each selected region,

analyzing the power spectral density (PSD) of the noise in said selected regions,

generating a parametric model of the noise present in the digital image based on said PSD of the noise in said selected regions, wherein said generating a parametric model of the noise present in the digital image based on said PSD for each selected region comprises;

taking a discrete Fourier transform of each noise sample, and

squaring each resulting frequency component to produce a periodogram,

averaging together the periodograms; and

selecting a best-fit linear combination of a small set of predefined PSDs.

2. The method of claim 1, wherein said identifying regions of approximately constant pixel value within the digital image comprises:

defining a plurality of regions in the digital image;

discarding any of said defined regions in which pixel values are saturated or clipped to generate a set of remaining regions;

analyzing said remaining regions to determine variation among pixel values within each remaining region; and

identifying said remaining regions having lower variation among pixel value.

3. The method of claim 2, wherein said defining a plurality of regions comprises allowing a user to select said regions.

4. The method of claim 2, wherein said defining a plurality of regions comprises:

setting a region size for each of said plurality of regions; and

using a random sampling pattern such that said plurality of regions include approximately all of the pixels in the digital image.

5. The method of claim 4, wherein said region size for each of said plurality of regions is between 16 and 32 pixels square.

6. The method of claim 2, wherein said discarding any of said defined regions in which pixel values are saturated or clipped comprises:

computing the average and standard deviation of all pixels in all samples; and

discarding those regions having an average value that is greater than one standard deviation from the overall average.

7. The method of claim 1, wherein said identifying regions of approximately constant pixel value comprises allowing a user to select said regions.

8. The method of claim 1, wherein said predetermined number of identified regions comprises between 5 and 10 regions.

9. The method of claim 1, wherein said predetermined number of identified regions comprises 7 regions.

10. The method of claim 1, wherein said analyzing the power spectral density of the noise in said selected regions comprises:

applying an auto correlation function (ACF) model to the noise present in each selected region; and

applying a discrete Fourier transform to the ACF model for each selected region to obtain a PSD for the noise in each selected region.

11. The method of claim 1, wherein the pixel value is a vector that includes multiple values or channels and each channel is analyzed independently.

12. The method of claim 11, further comprising using a decorrelating transform to uncorrelate any channels that are initially correlated.

13. The method of claim 1, wherein the pixel value is a vector that includes multiple values or channels and each channel is analyzed independently and simultaneously.

14. The method of claim 1, wherein the digital image comprises a plurality of frames and each frame is analyzed independently and simultaneously.

15. The method of claim 1, further comprising performing a homomorphic transformation on each pixel value prior to the analysis and a reverse transformation following the analysis.

16. A computer readable medium containing computer executable instructions for analyzing noise content in a digital image, which, when operating in a processor cause the processor to perform the functions of:

identifying regions of approximately constant pixel value within the digital image;

selecting a predetermined number of said identified regions based on pixel value variances within each said identified region;

analyzing said selected regions to generate a mathematical model of noise present in the digital image, wherein said analyzing said selected regions to generate a mathematical model of the noise present in the digital image comprises;

processing each selected region to remove a mean pixel value from each pixel in the selected region and obtain an approximation of the noise in each selected region,

analyzing the power spectral density (PSD) of the noise in said selected regions,

generating a parametric model of the noise present in the digital image based on said PSD of the noise in said selected regions, wherein said generating a parametric model of the noise present in the digital image based on said PSD for each selected region comprises;

taking a discrete Fourier transform of each noise sample, and

squaring each resulting frequency component to produce a periodogram,

averaging together the periodograms; and

selecting a best-fit linear combination of a small set of predefined PSDs.

17. The computer readable medium of claim 16, wherein said identifying regions of approximately constant pixel value within the digital image comprises:

defining a plurality of regions in the digital image;

discarding any of said defined regions in which pixel values are saturated or clipped to generate a set of remaining regions;

analyzing said remaining regions to determine variation among pixel value within each remaining region; and

identifying said remaining regions having lower variation among pixel value.

18. The computer readable medium of claim 17, wherein said defining a plurality of regions comprises:

setting a region size of between 16 and 32 pixels square for each of said plurality of regions; and

using a random sampling pattern such that said plurality of regions include approximately all of the pixels of the digital image.

19. The computer readable medium of claim 17, wherein said discarding any of said defined regions in which pixel values are saturated or clipped comprises:

computing the average and standard deviation of all pixels in all samples; and

discarding those regions having an average value that is greater than one standard deviation from the overall average.

20. The computer readable medium of claim 16, wherein said predetermined number of identified regions comprises between 5 and 10 regions.

21. The computer readable medium of claim 16, wherein said analyzing the power spectral density of the noise in said selected regions comprises:

applying an auto correlation function (ACF) model to the noise present in each selected region; and

applying a discrete Fourier transform to the ACF model for each selected region to obtain a PSD for the noise in each selected region.

22. The computer readable medium of claim 16, wherein the pixel value is a vector that includes multiple values or channels and each channel is analyzed independently.

23. A software program product for use with a processor, the software program product comprising processor readable program code for analyzing noise content in a digital image, wherein when said processor readable program code is executed in a processor, said processor readable program code causes said processor to perform the functions of:

identifying regions of approximately constant pixel value within the digital image;

selecting a predetermined number of said identified regions based on pixel value variances within each said identified region;

analyzing said selected regions to generate a mathematical model of noise present in the digital image, wherein said analyzing said selected regions to generate a mathematical model of the noise present in the digital image comprises;

processing each selected region to remove a mean pixel value from each pixel in the selected region and obtain an approximation of the noise in each selected region,

analyzing the power spectral density (PSD) of the noise in said selected regions,

generating a parametric model of the noise present in the digital image based on said PSD of the noise in said selected regions, wherein said generating a parametric model of the noise present in the digital image based on said PSD for each selected region comprises;

taking a discrete Fourier transform of each noise sample, and

squaring each resulting frequency component to produce a periodogram,

averaging together the periodograms; and

selecting a best-fit linear combination of a small set of predefined PSI)s.

24. The software program product of claim 23, wherein said identifying regions of approximately constant pixel value within the digital image comprises:

defining a plurality of regions in the digital image;

discarding any of said defined regions in which pixel values are saturated or clipped to generate a set of remaining regions;

analyzing said remaining regions to determine variation among pixel value within each remaining region; and

identifying said remaining regions having lower variation among pixel value.

25. The software program product of claim 24, wherein said defining a plurality of regions comprises:

setting a region size for each of said plurality of regions; and

using a random sampling pattern such that said plurality of regions include approximately all of the pixels in the digital image.

26. The software program product of claim 25, wherein said region size for each of said plurality of regions is between 16 and 32 pixels square.

27. The software program product of claim 24, wherein said discarding any of said defined regions in which pixel values are saturated or clipped comprises:

computing the average and standard deviation of all pixels in all sample, and

discarding those regions having an average value that is greater than one standard deviation from the overall average.

28. The software program product of claim 23, wherein said identifying regions of approximately constant pixel value comprise allowing a user to select said regions.

29. The software program product of claim 23, wherein said predetermined number of identified regions comprises between 5 and 10 regions.

30. The software program product of claim 23, wherein said predetermined number of identified regions comprises 7 regions.

31. The software program product of claim 23, wherein said analyzing the power spectral density of the noise in said selected regions comprises:

applying an auto correlation function (ACF) model to the noise present in each selected region; and

applying a discrete Fourier transform to the ACF model for each selected region to obtain a PSD for the noise in each selected region.

32. The software program product of claim 23, wherein said analyzing the power spectral density of the noise in said selected regions comprises:

applying an auto correlation function (ACF) model to the noise present in each selected region; and

applying a discrete Fourier transform to the ACF model for each selected region to obtain a PSD for the noise in each selected region.

33. The software program product of claim 23, wherein the pixel value is a vector that includes multiple values or channels and each channel is analyzed independently.

34. The software program product of claim 33, further comprising using a decorrelating transform to uncorrelate any channels that are initially correlated.

35. The software program product of claim 23, wherein the pixel value is a vector that includes multiple values or channels and each channel is analyzed independently and simultaneously.

36. The software program product of claim 23, further comprising:

performing a homomorphic transformation on each pixel value prior to the analysis and a reverse transformation following the analysis.

37. A method of analyzing noise content in a digital image, the method comprising:

identifying regions of approximately constant pixel value within the digital image, wherein said identifying comprises;

defining a plurality of regions in the digital image;

discarding any of said defined regions in which pixel values are saturated or clipped to generate a set of remaining regions;

analyzing said remaining regions to determine variation among pixel value within each remaining region; and

identifying said remaining regions having lower variation among pixel value;

selecting a predetermined number of said identified regions based on pixel value variances within each said identified region;

analyzing said selected regions to generate a mathematical model of noise present in the digital image, wherein said analyzing said selected regions to generate a mathematical model of the noise present in the digital image comprises;

processing each selected region to remove a mean pixel value from each pixel in the selected region and obtain an approximation of the noise in each selected region,

analyzing the power spectral density (PSD) of the noise in said selected regions,

generating a parametric model of the noise present in the digital image based on said PSD of the noise in said selected regions, wherein said generating a parametric model of the noise present in the digital image based on said PSD for each selected region comprises;

taking a discrete Fourier transform of each noise sample, and

squaring each resulting frequency component to produce a periodogram,

averaging together the periodograms; and

selecting a best-fit linear combination of a small set of predefined PSDs.

38. The method of claim 37, wherein said defining a plurality of regions comprises:

setting a region size for each of said plurality of regions; and

using a random sampling pattern such that said plurality of regions include approximately all of the pixels in the digital image.

39. The method of claim 37, wherein said predetermined number of identified regions comprises between 5 and 10 regions.

40. The method of claim 37, wherein said analyzing the power spectral density of the noise in said selected regions comprises:

applying an auto correlation function (ACF) model to the noise present in each selected region; and

applying a discrete Fourier transform to the ACF model for each selected region to obtain a PSD for the noise in each selected region.