Frequency-domain denoising

• US 10,102,613 B2
• Filed: 09/25/2014
• Issued: 10/16/2018
• Est. Priority Date: 09/25/2014
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Kopp, Timothy, Han, Jingning
• Primary Examiner(s): Czekaj, Dave
• Assistant Examiner(s): Sullivan, Tyler W.
• Application Number:
  US14/496,563
• Publication Number:
  US 20160094843A1
• Time in Patent Office: 1,482 Days
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20021 Dividing image into blocks, subimages or windows
• CPC Class Codes: G06T 2207/20052 Discrete cosine transform [DCT]
• CPC Class Codes: G06T 5/10 using non-spatial domain filtering
• CPC Class Codes: G06T 5/70 Denoising; Smoothing
• CPC Class Codes: H04N 19/117 Filters, e.g. for pre-processing or post-processing sub-band filter banks H04N19/635
• CPC Class Codes: H04N 19/154 Measured or subjectively estimated visual quality after decoding, e.g. measurement of distortion use of rate-distortion criteria H04N19/147
• CPC Class Codes: H04N 19/176 the region being a block, e.g. a macroblock
• CPC Class Codes: H04N 19/18 the unit being a set of transform coefficients
• CPC Class Codes: H04N 19/182 the unit being a pixel
• CPC Class Codes: H04N 19/192 the adaptation method, adaptation tool or adaptation type being iterative or recursive
• CPC Class Codes: H04N 19/48 using compressed domain processing techniques other than decoding, e.g. modification of transform coefficients, variable length coding [VLC] data or run-length data motion estimation in a transform domain H04N19/547; processing of decoded motion vectors H04N19/513
• CPC Class Codes: H04N 19/503 involving temporal prediction adaptive coding with adaptive selection between spatial and temporal predictive coding H04N19/107; adaptive coding with adaptive selection among a plurality of temporal predictive coding modes H04N19/109

Abstract

Encoding and decoding using frequency-domain denoising are provided. Encoding and decoding using frequency-domain denoising may include identifying a current source block from a current source frame from a source video stream, generating a prediction block for the current source block, generating a source transform block by transforming the current source block into the frequency domain, generating a prediction transform block by transforming the prediction block into the frequency domain, generating a denoised transform block based on the source transform block and the prediction transform block, generating a denoised block by inverse transforming the denoised transform block, generating a residual block based on a difference between the denoised block and the prediction block, generating an encoded block by encoding the residual block, and outputting or storing the encoded block.

Claims

1. A method comprising:

identifying a current source block from a current source frame from a source video stream;

generating a prediction block for the current source block;

generating a source transform block of the current source block by transforming the current source block into the frequency domain;

generating a prediction transform block of the prediction block by transforming the prediction block into the frequency domain;

generating, by a processor in response to instructions stored on a non-transitory computer readable medium, a denoised transform block based on an approximation of an integration of a product of the denoised transform block with a conditional probability function of the denoised transform block given the source transform block and the prediction transform block;

generating a denoised block by inverse transforming the denoised transform block;

generating a residual block based on a difference between the denoised block and the prediction block;

generating an encoded block by encoding the residual block; and

outputting or storing the encoded block.

2. The method of claim 1, wherein the current source block includes noise.

3. The method of claim 1, wherein generating the source transform block by transforming the current source block into the frequency domain includes applying a block-based transform to the current source block, such that the source transform block includes a plurality of source transform coefficients, and wherein generating the prediction transform block by transforming the prediction block into the frequency domain includes applying the block-based transform to the prediction block, such that the prediction transform block includes a plurality of prediction transform coefficients.

4. The method of claim 3, wherein the block-based transform is a discrete cosine transform.

5. The method of claim 1, wherein generating the denoised transform block includes denoising the source transform block based on the prediction transform block.

6. The method of claim 1, wherein generating the denoised transform block includes applying a frequency-domain cutoff.

7. The method of claim 6, wherein applying the frequency-domain cutoff includes applying a block-based frequency-domain cutoff.

8. The method of claim 1, wherein generating the denoised block includes applying a pixel-domain cutoff.

9. The method of claim 8, wherein applying the pixel-domain cutoff includes applying a block-based pixel-domain cutoff.

10. A method comprising:

identifying a current source block from a current source frame from a source video stream;

generating, by a processor in response to instructions stored on a non-transitory computer readable medium, an encoded block by encoding the current source block using frequency-domain denoising such that input noise in the current source block is reduced, wherein frequency-domain denoising includes;

generating a first residual block based on a difference between a denoised block corresponding to the current source block and a prediction block, wherein no second residual block is generated based on a difference between the current source block and any prediction block, by;

generating the prediction block for the current source block;

generating a source transform block by transforming the current source block into the frequency domain;

generating a prediction transform block by transforming the prediction block into the frequency domain;

generating a denoised transform block by integrating a product of the denoised transform block with a conditional probability function of the denoised transform block given the source transform block and the prediction transform block; and

generating the denoised block by inverse transforming the denoised transform block; and

generating the encoded block by encoding the first residual block; and

outputting or storing the encoded block.

11. The method of claim 10, wherein the current source block includes noise.

12. The method of claim 10, wherein generating the source transform block by transforming the current source block into the frequency domain includes applying a discrete cosine transform to the current source block, such that the source transform block includes a plurality of source transform coefficients, and wherein generating the prediction transform block by transforming the prediction block into the frequency domain includes applying the discrete cosine transform to the prediction block, such that the prediction transform block includes a plurality of prediction transform coefficients.

13. The method of claim 10, wherein generating the denoised transform block includes denoising the source transform block based on the prediction transform block.

14. The method of claim 10, wherein generating the denoised transform block includes applying a coefficient-based frequency-domain cutoff, applying a block-based frequency-domain cutoff, applying a pixel-based pixel-domain cutoff, or applying a block-based pixel-domain cutoff.

15. The method of claim 10, wherein encoding the current source block using frequency-domain denoising omits loop filtering.

16. The method of claim 15, further comprising:

generating a reconstructed block based on the encoded block, wherein generating the reconstructed block based on the encoded block includes loop filtering.

17. A method comprising:

identifying a current source block from a current source frame from a source video stream;

generating a prediction block for the current source block;

generating, a source transform block for the current source block by transforming the current source block into the frequency domain, wherein the source transform block is a first sum of a denoised transform block and a noise;

generating a prediction transform block for the prediction block by transforming the prediction block into the frequency domain, wherein the prediction transform block is a second sum of a noiseless prediction transform block and innovation;

generating, by a processor in response to instructions stored on a non-transitory computer readable medium the denoised transform block based on an integral of a model of a conditional probability density function of the noise and a model of a conditional probability density function of the innovation;

generating a denoised block by inverse transforming the denoised transform block;

generating a residual block based on a difference between the denoised block and the prediction block;

generating an encoded block by encoding the residual block; and

outputting or storing the encoded block.

18. The method of claim 17, wherein generating the denoised transform block includes applying a coefficient-based frequency-domain cutoff, applying a block-based frequency-domain cutoff, applying a pixel-based pixel-domain cutoff, or applying a block-based pixel-domain cutoff.