FREQUENCY-DOMAIN DENOISING

• US 20160094843A1
• Filed: 09/25/2014
• Published: 03/31/2016
• Est. Priority Date: 09/25/2014
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Han, Jingning, Kopp, Timothy
• Granted Patent: US 10,102,613 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20021 Dividing image into blocks, subimages or windows
• CPC Class Codes: G06T 2207/20052 Discrete cosine transform [DCT]
• CPC Class Codes: G06T 5/002 Denoising; Smoothing noise processing or correction adapted to be used in an image pickup device containing and electronic image sensor H04N23/81, H04N25/60 - H04N25/67
• CPC Class Codes: G06T 5/10 using non-spatial domain filtering
• CPC Class Codes: G06T 5/70 Denoising; Smoothing
• CPC Class Codes: H04N 19/117 Filters, e.g. for pre-processing or post-processing sub-band filter banks H04N19/635
• CPC Class Codes: H04N 19/154 Measured or subjectively estimated visual quality after decoding, e.g. measurement of distortion use of rate-distortion criteria H04N19/147
• CPC Class Codes: H04N 19/176 the region being a block, e.g. a macroblock
• CPC Class Codes: H04N 19/18 the unit being a set of transform coefficients
• CPC Class Codes: H04N 19/182 the unit being a pixel
• CPC Class Codes: H04N 19/192 the adaptation method, adaptation tool or adaptation type being iterative or recursive
• CPC Class Codes: H04N 19/48 using compressed domain processing techniques other than decoding, e.g. modification of transform coefficients, variable length coding [VLC] data or run-length data motion estimation in a transform domain H04N19/547; processing of decoded motion vectors H04N19/513
• CPC Class Codes: H04N 19/503 involving temporal prediction adaptive coding with adaptive selection between spatial and temporal predictive coding H04N19/107; adaptive coding with adaptive selection among a plurality of temporal predictive coding modes H04N19/109

Abstract

Encoding and decoding using frequency-domain denoising are provided. Encoding and decoding using frequency-domain denoising may include identifying a current source block from a current source frame from a source video stream, generating a predicted block for the current source block, generating a source transform block by transforming the current source block into the frequency domain, generating a predicted transform block by transforming the predicted block into the frequency domain, generating a denoised transform block based on the source transform block and the predicted transform block, generating a denoised block by inverse transforming the denoised transform block, generating a residual block based on a difference between the denoised block and the predicted block, generating an encoded block by encoding the residual block, and outputting or storing the encoded block.

Claims

1. A method comprising:

identifying a current source block from a current source frame from a source video stream;

generating a predicted block for the current source block;

generating a source transform block by transforming the current source block into the frequency domain;

generating a predicted transform block by transforming the predicted block into the frequency domain;

generating, by a processor in response to instructions stored on a non-transitory computer readable medium, a denoised transform block based on the source transform block and the predicted transform block;

generating a denoised block by inverse transforming the denoised transform block;

generating a residual block based on a difference between the denoised block and the predicted block;

generating an encoded block by encoding the residual block; and

outputting or storing the encoded block.

2. The method of claim 1, wherein the current source block includes noise.

3. The method of claim 1, wherein generating the source transform block by transforming the current source block into the frequency domain includes applying a block-based transform to the current source block, such that the source transform block includes a plurality of source transform coefficients, and wherein generating the predicted transform block by transforming the predicted block into the frequency domain includes applying the block-based transform to the predicted block, such that the predicted transform block includes a plurality of predicted transform coefficients.

4. The method of claim 3, wherein the block-based transform is a discrete cosine transform.

5. The method of claim 1, wherein generating the denoised transform block includes applying a denoising function to the source transform block based on the predicted transform block.

6. The method of claim 5, wherein applying the denoising function includes determining a result of an approximation of a conditional expectation of the denoised block based on the current source block and the predicted block.

7. The method of claim 1, wherein generating the denoised transform block includes applying a frequency-domain cutoff.

8. The method of claim 7, wherein applying the frequency-domain cutoff includes applying a block-based frequency-domain cutoff.

9. The method of claim 1, wherein generating the denoised block includes applying a pixel-domain cutoff.

10. The method of claim 9, wherein applying the pixel-domain cutoff includes applying a block-based pixel-domain cutoff.

11. A method comprising:

identifying a current source block from a current source frame from a source video stream;

generating, by a processor in response to instructions stored on a non-transitory computer readable medium, an encoded block by encoding the current source block using frequency-domain denoising; and

outputting or storing the encoded block.

12. The method of claim 11, wherein the current source block includes noise.

13. The method of claim 11, wherein encoding the current source block using frequency-domain denoising includes:

generating a predicted block for the current source block;

generating a source transform block by transforming the current source block into the frequency domain;

generating a predicted transform block by transforming the predicted block into the frequency domain;

generating a denoised transform block based on the source transform block and the predicted transform block;

generating a denoised block by inverse transforming the denoised transform block; and

generating a residual block based on a difference between the denoised block and the predicted block.

14. The method of claim 13, wherein generating the source transform block by transforming the current source block into the frequency domain includes applying a discrete cosine transform to the current source block, such that the source transform block includes a plurality of source transform coefficients, and wherein generating the predicted transform block by transforming the predicted block into the frequency domain includes applying the discrete cosine transform to the predicted block, such that the predicted transform block includes a plurality of predicted transform coefficients.

15. The method of claim 13, wherein generating the denoised transform block includes applying a denoising function to the source transform block based on the predicted transform block.

16. The method of claim 15, wherein applying the denoising function includes determining a result of an approximation of a conditional expectation of the denoised block based on the current source block and the predicted block.

17. The method of claim 13, wherein generating the denoised transform block includes applying a coefficient-based frequency-domain cutoff, applying a block-based frequency-domain cutoff, applying a pixel-based pixel-domain cutoff, or applying a block-based pixel-domain cutoff.

18. A method comprising:

identifying a current source block from a current source frame from a source video stream;

generating a predicted block for the current source block;

generating a source transform block by transforming the current source block into the frequency domain;

generating a predicted transform block by transforming the predicted block into the frequency domain;

generating, by a processor in response to instructions stored on a non-transitory computer readable medium, a denoised transform block by applying a denoising function to the source transform block based on the predicted transform block;

generating a denoised block by inverse transforming the denoised transform block;

generating a residual block based on a difference between the denoised block and the predicted block;

generating an encoded block by encoding the residual block; and

outputting or storing the encoded block.

19. The method of claim 18, wherein applying the denoising function includes determining a result of an approximation of a conditional expectation of the denoised block based on the current source block and the predicted block.

20. The method of claim 18, wherein generating the denoised transform block includes applying a coefficient-based frequency-domain cutoff, applying a block-based frequency-domain cutoff, applying a pixel-based pixel-domain cutoff, or applying a block-based pixel-domain cutoff.