Apparatus and method for decoding video encoded using a temporal filter

• US 8,665,952 B1
• Filed: 11/28/2011
• Issued: 03/04/2014
• Est. Priority Date: 09/15/2010
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Grange, Adrian, Wilkins, Paul, Galligan, Francis William
• Primary Examiner(s): AN, SHAWN S
• Application Number:
  US13/305,033
• Time in Patent Office: 827 Days
• Field of Search: 375/240.13, 375/240.25, 375/240.12, 375/240.29, 375/240.26, 382/233, 382/235, 382/238, 382/236, 382/260, 382/261
• US Class Current: 375/240.13
• CPC Class Codes: H04N 19/117 Filters, e.g. for pre-processing or post-processing sub-band filter banks H04N19/635
• CPC Class Codes: H04N 19/159 Prediction type, e.g. intra-frame, inter-frame or bidirectional frame prediction
• CPC Class Codes: H04N 19/176 the region being a block, e.g. a macroblock
• CPC Class Codes: H04N 19/194 involving only two passes
• CPC Class Codes: H04N 19/40 using video transcoding, i.e. partial or full decoding of a coded input stream followed by re-encoding of the decoded output stream
• CPC Class Codes: H04N 19/44 Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder
• CPC Class Codes: H04N 19/503 involving temporal prediction adaptive coding with adaptive selection between spatial and temporal predictive coding H04N19/107; adaptive coding with adaptive selection among a plurality of temporal predictive coding modes H04N19/109
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 19/63 using sub-band based transform, e.g. wavelets
• CPC Class Codes: H04N 19/80 Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation H04N19/635, H04N19/86 take precedence
• CPC Class Codes: H04N 19/85 using pre-processing or post-processing specially adapted for video compression
• CPC Class Codes: H04N 19/86 involving reduction of coding artifacts, e.g. of blockiness

Abstract

Embodiments of an apparatus and method for decoding an encoded bitstream to generate a video signal are taught. A decoder receives the bitstream and decodes a portion thereof to obtain at least a portion of a predictive reference frame. The reference frame is generated by, for example, selecting a filter set including a target frame with a block having a plurality of pixels and at least one adjacent frame, determining a coding mode associated with the block, determining a block-specific factor derived from the coding mode, determining weighted averages of pixels in the filter set and using the weighted averages to generate the predictive reference frame. The decoder also generates a residual from another portion of the encoded bitstream that represents a portion of a frame of the plurality of frames and reconstructs the portion of the frame by adding the residual to the predictive reference frame.

Claims

1. A method for decoding an encoded bitstream to generate a video signal having a plurality of frames, comprising:

receiving the encoded bitstream at a decoder, the encoded bitstream including the plurality of frames and a reference frame;

decoding a portion of the encoded bitstream to obtain at least a portion of the reference frame using a decoder, the at least a portion of the reference frame generated by a process of;

determining a plurality of factors for a first block residing in a selected frame and spatially correspondent to a portion of a reference frame, wherein each of the plurality of factors is at least partially derived from a correlation between the first block and at least a second block residing in a different one of the plurality of frames; and

applying a filter using filter weights formulated from the plurality of factors to the portion of the reference frame to generate the at least the portion of the reference frame;

generating a residual from another portion of the encoded bitstream, the residual representing a portion of a frame of the plurality of frames; and

reconstructing the portion of the frame by adding the residual to the at least the portion of the reference frame;

wherein the selected frame and the different one of the plurality of frames form a portion of a filter set of frames and the plurality of factors comprises;

a block-specific factor of the first block, the block-specific factor derived from a prediction coding mode associated with at least some blocks in the filter set, including the first block, and wherein determining the block-specific factor during the process comprises;

determining a first prediction error score for the first block relative to the second block;

determining a second prediction error score that is the best prediction error score for the first block relative to a block of pixels other than the second block; and

determining the block-specific factor using a ratio of the first prediction error score to the second error prediction score.

2. The method of claim 1 wherein the first and second blocks are spatially correspondent.

3. The method of claim 1 wherein the first block includes a plurality of pixels, and the plurality of factors further comprises:

for each of a plurality of pixels within the first block, a pixel-specific factor derived from the correlation that such pixel bears to a spatially correspondent pixel in a frame other than the selected frame; and

wherein the filter weights for each of a plurality of pixels within the portion of the reference frame were formulated from the pixel-specific factor and the block-specific factor of the first block.

4. The method of claim 1 wherein the block-specific factor comprises:

a value within a first range when the prediction coding mode is inter-frame coding without a motion vector;

a value within a second range when prediction coding mode is inter-frame coding with a motion vector; and

a value within a third range when prediction coding mode is the intra-frame coding;

wherein the first range, second range and third range do not completely overlap with each other.

5. An apparatus for decoding an encoded bitstream to generate a video signal having a plurality of frames, comprising:

a decoder configured to;

receive the encoded bitstream;

decode a portion of the encoded bitstream to obtain at least a portion of a predictive reference frame using the decoder, the at least a portion of the predictive reference frame generated by a process of;

determining a plurality of factors for a first block residing in a selected frame and spatially correspondent to a portion of a reference frame, wherein each of the plurality of factors is at least partially derived from a correlation between the first block and at least a second block residing in a different one of the plurality of frames; and

applying a filter using filter weights formulated from the plurality of factors to the portion of the reference frame to generate the at least the portion of the predictive reference frame;

generate a residual from another portion of the encoded bitstream, the residual representing a portion of a frame of the plurality of frames; and

reconstruct the portion of the frame by adding the residual to the at least the portion of the predictive reference frame; and

a display for displaying the video signal;

wherein the plurality of factors includes;

a block-specific factor for the first block, the block-specific factor derived from a prediction coding mode for the first block, the prediction coding mode being one of inter-frame coding without motion compensation, inter-frame coding with motion compensation or intra-frame coding; and

wherein the block-specific factor comprises;

a value within a first range when the prediction coding mode is the inter-frame coding without motion compensation;

a value within a second range when prediction coding mode is the inter-frame coding with motion compensation; and

a value within a third range when prediction coding mode is the intra-frame coding;

wherein the first range, second range and third range do not completely overlap with each other.

6. The apparatus of claim 5 wherein the correlation is a temporal correlation, the selected frame and the different one of the plurality of frames form a portion of a filter set, and a size of the filter set is based upon the temporal correlation.

7. The apparatus of claim 5 wherein the decoder is configured to:

generate a second residual from another portion of the encoded bitstream, the second residual representing a portion of a second frame of the plurality of frames; and

reconstruct the portion of the second frame by adding the second residual to the at least the portion of the predictive reference frame.

8. The apparatus of claim 5 wherein the portion of the encoded bitstream includes header information.

9. The apparatus of claim 5 wherein determining the block-specific factor during the process comprises:

determining a first prediction error score for the first block relative to the second block;

determining a second prediction error score that is the best prediction error score for the first block relative to a block of pixels other than the second block; and

determining the block-specific factor using a ratio of the first prediction error score to the second error prediction score.

10. A method for decoding an encoded bitstream to generate a video signal having a plurality of frames, comprising:

receiving the encoded bitstream at a decoder;

decoding a portion of the encoded bitstream to obtain at least a portion of a predictive reference frame using the decoder, the at least a portion of the predictive reference frame generated by a process of;

determining a plurality of factors for a first block residing in a selected frame, wherein each of the plurality of factors is at least partially indicative of a correlation between the first block and at least a second block residing in a different one of the plurality of frames, wherein the plurality of factors comprises a prediction coding mode associated with the first block and a block-specific factor of the first block, the block-specific factor derived from the prediction coding mode and comprising a value within a first range when the prediction coding mode is inter-frame coding without a motion vector, a value within a second range when prediction coding mode is inter-frame coding with a motion vector, and a value within a third range when prediction coding mode is intra-frame coding, and wherein the first range, second range and third range do not completely overlap with each other; and

applying filter weights derived from the plurality of factors to at least the first block and the second block to generate the at least the portion of the predictive reference frame;

generating a residual from another portion of the encoded bitstream, the residual representing a portion of a frame of the plurality of frames; and

reconstructing the portion of the frame by adding the residual to the at least the portion of the predictive reference frame.

11. The method of claim 10 wherein the first block includes a plurality of pixels, and the plurality of factors further comprises:

for each of a plurality of pixels within the first block, a pixel-specific factor derived from the correlation that such pixel bears to a spatially correspondent pixel in a frame other than the selected frame; and

wherein the filter weights for each of the plurality of pixels within the first block were derived from the pixel-specific factor and the block-specific factor of the first block.

12. The method of claim 10 wherein determining the block-specific factor during the process comprises:

determining a first prediction error score for the first block relative to the second block;

determining a second prediction error score that is the best prediction error score for the first block relative to a block of pixels other than the second block; and

determining the block-specific factor using a ratio of the first prediction error score to the second error prediction score.