System for manipulating noise in digital images

• US 20020034337A1
• Filed: 05/23/2001
• Published: 03/21/2002
• Est. Priority Date: 05/23/2000
• Status: Active Grant

• Current Assignee: Adobe Inc.
• Original Assignee: Adobe Systems Incorporated (Adobe Inc.)
• Inventors: Shekter, Jonathan Martin
• Granted Patent: US 6,990,252 B2
• Time in Patent Office: Days
• US Class Current: 382/275
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/20016 Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform
• CPC Class Codes: G06T 2207/20076 Probabilistic image processing
• CPC Class Codes: G06T 2207/20204 Removing film grain; Adding simulated film grain
• CPC Class Codes: G06T 5/10 using non-spatial domain filtering
• CPC Class Codes: G06T 5/20 using local operators
• CPC Class Codes: G06T 5/70 Denoising; Smoothing
• CPC Class Codes: G06V 10/30 Noise filtering

Abstract

An apparatus for analyzing the broadband noise content of a digital image comprising the following: a means of automatically identifying regions of originally constant color in the image by analysis of the variance of pixel values of regions of the image; a means of automatically detecting and discarding regions deemed to be unrepresentative of the true noise content of an image, including under- and over- exposed regions; a means of allowing the user to manually select some or all required constant color regions if desired; and, a means of analyzing such constant color regions to generate a parametric or non-pararnetric model of the noise in the image, including frequency characteristic within and between channels and other characteristics such as phase which might describe structured noise.

Claims

1. Apparatus for analyzing the broadband noise content of a digital image, comprising

means of automatically identifying regions of originally constant color in the image by analysis of the variance of pixel values of regions of the image;

means of automatically detecting and discarding regions deemed to be unrepresentative of the true noise content of an image, including under- and over-exposed regions;

means of allowing the user to manually select some or all required constant color regions if desired;

means of analyzing such constant color regions to generate a parametric or non-parametic model of the noise in the image, including frequency characteristic within and between channels and other characteristics such as phase which might describe structured noise.

2. Apparatus for modifying an image by reducing the amount of broadband noise present in the image, comprising:

means for computing an over-complete transformation of the image into representative coefficients, such that this transform is invertible and each coefficient is localized both in space and frequency, and which optionally operates between multiple frames if present of an image sequence;

means for computing the variance due to noise in each such coefficient using a spectral model of the noise in the source image;

means for modifying such coefficients to effect noise reduction based on Bayesian estimation or statistical estimation in general, such estimation performed simultaneously across multiple color channels if present;

means for possibly applying fltering which operates between frames (temporal filtering) after intra-frame noise reduction has been performed to achieve farther noise reduction;

means for possibly applying additional sharpening or contrast enhancement filters as a post-processing step.

3. Apparatus for adding noise to a source image, comprising

means for allowing the user to specify some number of characteristics of the noise to be added;

means for allowing the user to adjust the color balance and relative correlation of the noise to be added;

means for allowing the user to adjust the frequency characteristics of the generated noise, including controls which ultimately modify the phase relationships of different frequency components so that various types of structured noise may be created.

4. Apparatus for adding noise to a source image such that the total noise in the resulting composite image matches the noise in a reference image, with or without user specified modifications, comprising

means for using extracted reference noise samples to generate 2D or 3D convolution kernels capable of resynthesizing the reference noise when applied to a white noise field;

means for modifying such kernels to compensate for existing noise in the source image so that a match is maintained regardless of the ciurent noise level in the source image;

means for analyzing the color relationship of noise in a reference image, including amount of noise in each channel and inter-channel correlations of all kinds, and replicating these color relations in a source image.

5. Apparatus for combining generated or extracted noise with a source image, comprising

one or several of different image processing operations which can be used to combine generated or extracted noise with a source image;

a combination mode which mimics the appearance of film grain by inverting the pixel values of the source image, multiplying them by the generated noise values after they have undergone exponentiation, then inverting the modified pixels again;

means to allow the user to select the desired mode if more than one is available.