Apparatus and methodology for a video codec system with noise reduction capability

• US 8,885,706 B2
• Filed: 09/16/2011
• Issued: 11/11/2014
• Est. Priority Date: 09/16/2011
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Bankoski, James Barnard, Kokaram, Anil Christopher, Xu, Yaowu
• Primary Examiner(s): Anyikire, Chikaodili E
• Application Number:
  US13/234,614
• Publication Number:
  US 20130128957A1
• Time in Patent Office: 1,152 Days
• US Class Current: 375/240.03
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20021 Dividing image into blocks, subimages or windows
• CPC Class Codes: G06T 5/70 Denoising; Smoothing
• CPC Class Codes: H04N 19/124 Quantisation
• CPC Class Codes: H04N 19/126 Details of normalisation or weighting functions, e.g. normalisation matrices or variable uniform quantisers
• CPC Class Codes: H04N 19/132 Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
• CPC Class Codes: H04N 19/136 Incoming video signal characteristics or properties
• CPC Class Codes: H04N 19/14 Coding unit complexity, e.g. amount of activity or edge presence estimation H04N19/146 takes precedence
• CPC Class Codes: H04N 19/176 the region being a block, e.g. a macroblock
• CPC Class Codes: H04N 19/18 the unit being a set of transform coefficients
• CPC Class Codes: H04N 19/423 characterised by memory arrangements H04N19/433 takes precedence
• CPC Class Codes: H04N 19/46 Embedding additional information in the video signal during the compression process H04N19/517, H04N19/68, H04N19/70 take precedence
• CPC Class Codes: H04N 19/48 using compressed domain processing techniques other than decoding, e.g. modification of transform coefficients, variable length coding [VLC] data or run-length data motion estimation in a transform domain H04N19/547; processing of decoded motion vectors H04N19/513

Abstract

Systems and methods for noise reduction are disclosed herein. The system includes a video codec system that can encode and/or decode video information. A noise representation component can identify flat regions and a quantizer can utilize the identified flat regions to suppress noise during compression. By suppressing noise during compression, the size of the video file to be compressed can be reduced, compression can use less resources and take less time, and the speed at which the compressed information is transferred can benefit. Noise can be reintroduced during the reconstruction of the video. Accordingly, both noise reduction and noise synthesis can be accomplished.

Claims

1. A media codec system, comprising:

a memory that stores computer executable components; and

a processor that executes the following computer executable components stored within the memory;

a noise representation component configured to identify a flat image region having pixel values with differences below a variance threshold, and to generate a noise energy spectrum by applying a transform to the flat image region to generate a first set of transform coefficients, the noise energy spectrum represented by the first set of transform coefficients;

a quantization component configured to adaptively suppress noise associated with one or more blocks of an input image frame by changing a dead-zone of quantization based on the noise energy spectrum; and

an entropy encoding component configured to receive quantized transform coefficients from the quantization component and to encode the quantized transform coefficients into a bitstream for transmission.

2. The system of claim 1, wherein the noise representation is configured to identify the flat image region within one or more input frames.

3. The system of claim 1, wherein the flat image region is identified based on comparison of variances in pixel values of a chroma or a luma image block.

4. The system of claim 1, wherein the noise representation component is configured to generate a first noise energy spectrum for a first flat image region and a second noise energy spectrum for a second flat image region, the first flat image region having pixel luma values with differences below a first variance threshold and the second flat image region having pixel chroma values with differences below a second variance threshold;

and wherein the quantization component is configured to adaptively suppress noise associated with the one or more blocks by changing the dead-zone of quantization for luma blocks based on the first noise energy spectrum and changing the dead-zone of quantization for chroma blocks based on the second noise energy spectrum.

5. The system of claim 1, wherein the noise energy spectrum is associated with several flat image regions.

6. The system of claim 1, wherein the noise representation component utilizes calibration information to generate the noise energy spectrum.

7. The system of claim 6, wherein the calibration information is based on equipment noise.

8. The system of claim 6, wherein the calibration information is based on a flat background image.

9. The system of claim 1, wherein the quantization component adaptively changes the dead-zone of quantization of a second set of transform coefficients.

10. The system of claim 9, wherein the quantization component uses the first set of transform coefficients to define threshold values associated with the dead-zone of the second set of transform coefficients.

11. The system of claim 10, wherein the threshold values are a function of the values of the first set of transform coefficients.

12. The system of claim 11, wherein the threshold values are equal to the values of the first set of transform coefficients.

13. The system of claim 1, wherein the noise representation component generates a first noise energy spectrum based on variances in luma components and a second noise energy spectrum based on variances in chroma components.

14. A media codec system, comprising:

a memory that stores computer executable components; and

a microprocessor that executes the following computer executable components stored within the memory;

an entropy decoding component configured to receive a bit stream and generate header information and sets of quantized coefficients from the bit stream;

a de-quantization component configured to de-quantize the sets of quantized coefficients and synthesize noise into the sets of quantized coefficients to generate sets of de-quantized coefficients including the noise; and

an inverse transformation component configured to receive the sets of de-quantized coefficients including the noise and inverse transform the sets of de-quantized coefficients to generate decompressed media data.

15. The system of claim 14, wherein the header information comprises noise energy information, the noise energy information represented by a first set of transform coefficients generated by applying a transform to a flat image region, the flat image region having pixel values with differences below a variance threshold.

16. The system of claim 15, wherein the de-quantization component employs the noise energy information to synthesize noise into the sets of quantized coefficients by using each transform coefficient of the flat image region to define a respective threshold for a dead-zone of quantization to be applied to a spatially-corresponding quantized coefficient of the sets of quantized coefficients.

17. A media compression method, comprising:

employing a microprocessor to execute computer executable instructions stored in a memory to perform the following acts;

generating a noise energy spectrum by applying a transform to the flat image region to generate a first set of transform coefficients, the flat image region having pixel values with differences below a variance threshold and the noise energy spectrum represented by the first set of transform coefficients;

adaptively changing a set of dead-zone values of quantization for each quantized coefficient of a block of video image data according to a spatially-corresponding transform coefficient of the first set of transform coefficients;

using the set of dead-zone values of quantization to generate a second set of transform coefficients associated with compressed image data so as to reduce noise within the block; and

entropy encoding the quantized second set of transform coefficients for inclusion in a bitstream.

18. The method of claim 17, wherein the dead-zone values are set to be equal to the values of the first set of transform coefficients.

19. The method of claim 17, wherein the noise energy spectrum is generated by using one or more of calibration information, equipment noise or a flat background image.

20. The method of claim 17, further comprising generating a plurality of noise energy spectrums and corresponding sets of dead-zone values of quantization.

21. A media decompression method, comprising:

employing a microprocessor to execute computer executable instruction stored in a memory to perform the following act;

using a set of dead-zone values of quantization to decompress compressed image data, wherein the dead-zone values correspond to a noise energy spectrum.

22. The method of claim 21, wherein the dead-zone values are read from the compressed image data and are generated by a combination of a quantization value of the compressed image data and a first set of transform coefficients generated by applying a transform to a flat image region, the flat image region having pixel values with differences below a variance threshold.

23. An apparatus comprising a computer readable medium encoding the following computer executable components:

a noise representation component configured to receive calibration information and video image data and to use the calibration information to generate a noise energy spectrum;

a quantization component configured to use the noise energy spectrum to generate a plurality of thresholds to selectively discard a portion of the video image data associated with noise for quantization, each of the plurality of thresholds based on a respective value within the noise energy spectrum; and

an entropy encoding component configured to receive quantized transform coefficients from the quantization component and to encode the quantized transform coefficients into a bitstream for transmission.

24. The apparatus of claim 23, wherein the noise representation component identifies a flat region associated with the video image data, the flat region having pixel values with difference below a variance threshold.

25. The apparatus of claim 23, wherein the noise energy spectrum is generated by applying a transform to a flat region to generate a set of transform coefficients as values of the noise energy spectrum, the flat region having pixel values with differences below a variance threshold.

26. The apparatus of claim 25, wherein each of the plurality of thresholds is applied to a spatially-corresponding quantized coefficient of sets of quantized coefficients forming blocks of a video frame of the video image data.