System for manipulating noise in digital images

• US 7,599,572 B2
• Filed: 08/05/2005
• Issued: 10/06/2009
• Est. Priority Date: 05/23/2000
• Status: Active Grant

• Current Assignee: Adobe Inc.
• Original Assignee: Adobe Systems Incorporated (Adobe Inc.)
• Inventors: Shekter, Jonathan Martin
• Primary Examiner(s): TUCKER, WESLEY J
• Application Number:
  US11/198,724
• Publication Number:
  US 20050276515A1
• Time in Patent Office: 1,523 Days
• Field of Search: 382/272, 382/273, 382/282, 382/283, 358/463, 702185-195, 702/199
• US Class Current: 382/272
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/20016 Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform
• CPC Class Codes: G06T 2207/20076 Probabilistic image processing
• CPC Class Codes: G06T 2207/20204 Removing film grain; Adding simulated film grain
• CPC Class Codes: G06T 5/10 using non-spatial domain filtering
• CPC Class Codes: G06T 5/20 using local operators
• CPC Class Codes: G06T 5/70 Denoising; Smoothing
• CPC Class Codes: G06V 10/30 Noise filtering

Abstract

An apparatus for analyzing the broadband noise content of a digital image comprising the following: a means of automatically identifying regions of originally constant color in the image by analysis of the variance of pixel values of regions of the image; a means of automatically detecting and discarding regions deemed to be unrepresentative of the true noise content of an image, including under- and over- exposed regions; a means of allowing the user to manually select some or all required constant color regions if desired; and, a means of analyzing such constant color regions to generate a parametric or non-parametric model of the noise in the image, including frequency characteristic within and between channels and other characteristics such as phase which might describe structured noise.

Claims

1. A method for analyzing noise content in a digital image, comprising:

using a computer to perform;

sampling regions of the digital image;

discarding sampled regions that do not satisfy a pre-specified condition, wherein said discarding sampled regions that do not satisfy the pre-specified condition involves;

discarding sampled regions that are underexposed or overexposed;

computing the variance of the pixel values in each remaining sampled region; and

discarding sampled regions where the variance is above a pre-specified value;

extracting noise information from each remaining sampled region; and

generating a mathematical model of the noise using the noise information.

2. The method of claim 1, further comprising using the mathematical model to add noise to a second digital image, so that the noise structure of the second digital image substantially matches the noise structure of the digital image.

3. The method of claim 2, further comprising compositing the second digital image (including the added noise) with digital image to form a single composite image.

4. The method of claim 1, wherein said discarding sampled regions that are underexposed or overexposed involves:

computing a mean and a standard deviation of the pixel values in all of the sampled regions; and

discarding sampled regions if the mean of the pixel values for the sampled region is greater than one standard deviation from the mean of the pixel values for all sampled regions.

7. The method of claim 1, wherein the sampled regions are squares;

and wherein the sampling pattern covers the entire digital image.

8. The method of claim 1, wherein the sampled regions are selected based on manual input received from a user.

9. The method of claim 1,

wherein for a digital image with multiple channels, the mathematical model of the noise is generated independently for each channel;

and

wherein if the noise is correlated between multiple channels, a decorrelating transform is applied to transform the image to a color-space where all channels are uncorrelated.

10. The method of claim 1, further comprising using the mathematical model of the noise to reduce the noise in the digital image.

11. The method of claim 1, further comprising using the mathematical model of the noise in the digital image to add noise to a second digital image such that the noise in the second digital image substantially matches the noise in the digital image.

5. A method for analyzing noise content in a digital image, comprising:

using a computer to perform;

sampling regions of the digital image;

discarding sampled regions that do not satisfy a pre-specified condition;

extracting noise information from each remaining sampled region, wherein said extracting noise information from each remaining sampled region involves;

for each remaining sampled region;

computing the mean of the pixel values for the remaining sampled region to generate a mean pixel value; and

subtracting the mean pixel value from the remaining sampled region, thereby generating sampled regions containing noise information, but no image content; and

generating a mathematical model of the noise using the noise information.

6. A method for analyzing noise content in a digital image, comprising:

using a computer to perform;

sampling regions of the digital image;

discarding sampled regions that do not satisfy a pre-specified condition;

extracting noise information from each remaining sampled region; and

generating a mathematical model of the noise using the noise information, wherein said generating a mathematical model of the noise using the noise information involves;

applying a discrete Fourier transform to each remaining sampled region and to an auto correlation function of the noise;

taking the square of the discrete Fourier transforms;

averaging the squares of the discrete Fourier transforms; and

selecting a best-fit linear-combination of the averages to generate the power spectral density of the noise for the digital image.

12. A computer-readable storage medium storing instructions that when executed by a computer cause the computer to perform a method for analyzing noise content in a digital image, the method comprising:

sampling regions of the digital image;

discarding sampled regions that do not satisfy a pre-specified condition, wherein said discarding sampled regions that do not satisfy the pre-specified condition involves;

discarding sampled regions that are underexposed or overexposed;

computing the variance of the pixel values in each remaining sampled region; and

discarding sampled regions where the variance is above a pre-specified value;

extracting noise information from each remaining sampled region; and

generating a mathematical model of the noise using the noise information.

13. The computer-readable storage medium of claim 12, wherein the method further comprises using the mathematical model to add noise to a second digital image, so that the noise structure of the second digital image substantially matches the noise structure of the digital image.

14. The computer-readable storage medium of claim 12, wherein the method further comprises compositing the second digital image (including the added noise) with digital image to form a single composite image.

15. The computer-readable storage medium of claim 12, wherein discarding sampled regions that are underexposed or overexposed involves:

computing a mean and a standard deviation of the pixel values in all of the sampled regions; and

discarding sampled regions if the mean of the pixel values for the sampled region is greater than one standard deviation from the mean of the pixel values for all sampled regions.

18. The computer-readable storage medium of claim 12, wherein the sampled regions are squares;

and wherein the sampling pattern covers the entire digital image.

19. The computer-readable storage medium of claim 12, wherein the sampled regions are selected based on manual input received from a user.

20. The computer-readable storage medium of claim 12,

wherein for a digital image with multiple channels, the mathematical model of the noise is generated independently for each channel;

and

wherein if the noise is correlated between multiple channels, a decorrelating transform is applied to transform the image to a color-space where all channels are uncorrelated.

21. The computer-readable storage medium of claim 12, wherein the method further comprises using the mathematical model of the noise to reduce the noise in the digital image.

22. The computer-readable storage medium of claim 12, wherein the method further comprises using the mathematical model of the noise in the digital image to add noise to a second digital image such that the noise in the second digital image substantially matches the noise in the digital image.

16. A computer-readable storage medium storing instructions that when executed by a computer cause the computer to perform a method for analyzing noise content in a digital image, the method comprising:

sampling regions of the digital image;

discarding sampled regions that do not satisfy a pre-specified condition;

extracting noise information from each remaining sampled region, wherein said extracting noise information from each remaining sampled region involves;

for each remaining sampled region;

computing the mean of the pixel values for the remaining sampled region to generate a mean pixel value; and

subtracting the mean pixel value from the remaining sampled region, thereby generating sampled regions containing noise information, but no image content; and

generating a mathematical model of the noise using the noise information.

17. A computer-readable storage medium storing instructions that when executed by a computer cause the computer to perform a method for analyzing noise content in a digital image, the method comprising:

sampling regions of the digital image;

discarding sampled regions that do not satisfy a pre-specified condition;

extracting noise information from each remaining sampled region; and

generating a mathematical model of the noise using the noise information, wherein said generating a mathematical model of the noise using the noise information involves;

applying a discrete Fourier transform to each remaining sampled region and to an auto correlation function of the noise;

taking the square of the discrete Fourier transforms;

averaging the squares of the discrete Fourier transforms; and

selecting a best-fit linear-combination of the averages to generate the power spectral density of the noise for the digital image.

23. An apparatus for analyzing noise content in a digital image, comprising:

a noise analysis mechanism configured to;

sample regions of the digital image;

discard sampled regions that do not satisfy a pre-specified condition;

wherein to discard sampled regions that do not satisfy the pre-specified condition, the noise analysis mechanism is further configured to;

discard sampled regions that are underexposed or overexposed;

compute the variance of the pixel values in each remaining sampled region; and

discard sampled regions where the variance is above a pre-specified value;

extract noise information from each remaining sampled region; and

generate a mathematical model of the noise using the noise information.