Compound motion-compensated prediction

• US 10,412,410 B2
• Filed: 08/14/2017
• Issued: 09/10/2019
• Est. Priority Date: 08/14/2017
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Han, Jingning, Xu, Yaowu
• Primary Examiner(s): Kalapodas, Dramos
• Application Number:
  US15/675,979
• Publication Number:
  US 20190052897A1
• Time in Patent Office: 757 Days
• CPC Class Codes: H04N 19/107 between spatial and temporal predictive coding, e.g. picture refresh
• CPC Class Codes: H04N 19/126 Details of normalisation or weighting functions, e.g. normalisation matrices or variable uniform quantisers
• CPC Class Codes: H04N 19/172 the region being a picture, frame or field
• CPC Class Codes: H04N 19/176 the region being a block, e.g. a macroblock
• CPC Class Codes: H04N 19/52 by predictive encoding
• CPC Class Codes: H04N 19/567 Motion estimation based on rate distortion criteria
• CPC Class Codes: H04N 19/573 Motion compensation with multiple frame prediction using two or more reference frames in a given prediction direction
• CPC Class Codes: H04N 19/577 Motion compensation with bidirectional frame interpolation, i.e. using B-pictures
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 19/70 characterised by syntax aspects related to video coding, e.g. related to compression standards
• CPC Class Codes: H04N 19/82 involving filtering within a prediction loop
• CPC Class Codes: H04N 19/86 involving reduction of coding artifacts, e.g. of blockiness
• CPC Class Codes: H04N 19/91 Entropy coding, e.g. variable length coding [VLC] or arithmetic coding
• CPC Class Codes: H04N 23/00 Cameras or camera modules comprising electronic image sensors; Control thereof

Abstract

A prediction scheme is selected for encoding or decoding a video block. A first compound motion block can be determined by weighting distances from a first reference frame to the video frame and from a second reference frame to the video frame using one or more quantized weighting coefficients. A second compound motion block can be determined based on an average of pixel values a video block of the first reference frame and pixel values from a video block of the second reference frame. One of the first compound motion block or the second compound motion block is selected and used to generate a prediction block. Alternatively, data encoded to a bitstream including the video frame can be used to determine which compound motion block to use to generate the prediction block. The current block of the video frame is then encoded or decoded using the prediction block.

Claims

1. A method for encoding a current block of a video frame, the method comprising:

determining a first compound motion block by;

determining a pair of quantized weighting coefficients based on a ratio of a first distance and a second distance, the first distance representing a distance from a first reference frame to the video frame, the second distance representing a distance from a second reference frame to the video frame, the pair of quantized weighting coefficients including a first quantized weighting coefficient and a second quantized weighting coefficient;

determining a first value by applying the first quantized weighting coefficient to pixel values from a video block of the first reference frame;

determining a second value by applying the second quantized weighting coefficient to pixel values from a video block of the second reference frame; and

determining the first compound motion block by dividing a sum of the first value and the second value by a sum of the first quantized weighting coefficient and the second quantized weighting coefficient;

determining a second compound motion block based on an average of the pixel values from the video block of the first reference frame and the pixel values from the video block of the second reference frame;

selecting one of the first compound motion block or the second compound motion block;

generating a prediction block using the selected one of the first compound motion block or the second compound motion block; and

encoding the current block using the prediction block.

2. The method of claim 1, wherein,

when the first distance is greater than the second distance, the first quantized weighting coefficient has a value of 1 and the second quantized weighting coefficient has a value of at least 1,

and, when the first distance is less than the second distance, the first quantized weighting coefficient has a value of at least 1 and the second quantized weighting coefficient has a value of 1.

3. The method of claim 1, wherein selecting the one of the first compound motion block or the second compound motion block comprises:

determining rate-distortion values resulting from predicting motion of the current block using motion vectors associated with each of the first compound motion block and the second compound motion block; and

selecting the one of the first compound motion block or the second compound motion block resulting in a lower one of the rate-distortion values.

4. The method of claim 1, further comprising:

encoding one or more syntax elements indicative of the selection of the first compound motion block or the second compound motion block to a bitstream to which the current block is encoded.

16. The method of claim 1, wherein, when the first distance is greater than the second distance, the ratio is a first ratio of the first distance over the second distance, and, when the first distance is not greater than the second distance, the ratio is a second ratio of the second distance over the first distance.

17. The method of claim 16, wherein, when the first ratio or the second ratio is less than 1.5, the first quantized weighting coefficient and the second quantized weighting coefficient are both equal to 1,

wherein, when the first ratio is greater than 1.5 and less than 2.5, the first quantized weighting coefficient is 1 and the second quantized weighting coefficient is 2,

wherein, when the first ratio is greater than 2.5 and less than 3.5, the first quantized weighting coefficient is 1 and the second quantized weighting coefficient is 3,

wherein, when the first ratio is greater than 3.5, the first quantized weighting coefficient is 1 and the second quantized weighting coefficient is 4,

wherein, when the second ratio is greater than 1.5 and less than 2.5, the first quantized weighting coefficient is 2 and the second quantized weighting coefficient is 1,

wherein, when the second ratio is greater than 2.5 and less than 3.5, the first quantized weighting coefficient is 3 and the second quantized weighting coefficient is 1,

wherein, when the second ratio is greater than 3.5, the first quantized weighting coefficient is 4 and the second quantized weighting coefficient is 1.

5. A method for decoding an encoded block of an encoded video frame, the method comprising:

determining, based on one or more syntax elements encoded to a bitstream including the encoded video frame, whether the encoded block was encoded by weighting distances from each of a plurality of reference frames to the encoded video frame;

responsive to determining that the encoded block was encoded by weighting the distances from each of the plurality of reference frames to the encoded video frame, determining a compound motion block by;

determining a pair of quantized weighting coefficients based on a ratio of a first distance and a second distance, the first distance representing a distance from a first reference frame of the plurality of reference frames to the encoded video frame, the second distance representing a distance from a second reference frame of the plurality of reference frames to the encoded video frame, the pair of quantized weighting coefficients including a first quantized weighting coefficient and a second quantized weighting coefficient;

determining a first value by applying the first quantized weighting coefficient to pixel values from a video block of the first reference frame;

determining a second value by applying the second quantized weighting coefficient to pixel values from a video block of the second reference frame; and

determining the first compound motion block by dividing a sum of the first value and the second value by a sum of the first quantized weighting coefficient and the second quantized weighting coefficient;

generating a prediction block using the compound motion block; and

decoding the encoded block using the prediction block.

6. The method of claim 5, wherein,

when the first distance is greater than the second distance, the first quantized weighting coefficient has a value of 1 and the second quantized weighting coefficient has a value of at least 1,

and, when the first distance is less than the second distance, that the first quantized weighting coefficient has a value of at least 1 and the second quantized weighting coefficient has a value of 1.

7. The method of claim 5, wherein the compound motion block is a first compound motion block and the prediction block is a first prediction block, the method further comprising:

responsive to determining that the encoded block was not encoded by weighting the distances using the one or more quantized weighting coefficients, determining a second compound motion block based on an average of pixel values from an encoded video block of the first reference frame and pixel values from an encoded video block of the second reference frame;

generating a second prediction block using the second compound motion block; and

decoding the encoded block using the second prediction block.

8. The method of claim 5, further comprising:

decoding the one or more syntax elements from the bitstream prior to determining whether the encoded block was encoded by weighting the distances from each of the plurality of reference frames to the encoded video frame.

9. The method of claim 5, wherein an order of encoded video frames of a video sequence including the encoded video frame, the first reference frame, and the second reference frame is indicated within the bitstream.

18. The method of claim 5, wherein, when the first distance is greater than the second distance, the ratio is a first ratio of the first distance over the second distance, and, when the first distance is not greater than the second distance, the ratio is a second ratio of the second distance over the first distance.

19. The method of claim 18, wherein, when the first ratio or the second ratio is less than 1.5, the first quantized weighting coefficient and the second quantized weighting coefficient are both equal to 1,

wherein, when the first ratio is greater than 1.5 and less than 2.5, the first quantized weighting coefficient is 1 and the second quantized weighting coefficient is 2,

wherein, when the first ratio is greater than 2.5 and less than 3.5, the first quantized weighting coefficient is 1 and the second quantized weighting coefficient is 3,

wherein, when the first ratio is greater than 3.5, the first quantized weighting coefficient is 1 and the second quantized weighting coefficient is 4,

wherein, when the second ratio is greater than 1.5 and less than 2.5, the first quantized weighting coefficient is 2 and the second quantized weighting coefficient is 1,

wherein, when the second ratio is greater than 2.5 and less than 3.5, the first quantized weighting coefficient is 3 and the second quantized weighting coefficient is 1,

wherein, when the second ratio is greater than 3.5, the first quantized weighting coefficient is 4 and the second quantized weighting coefficient is 1.

10. An apparatus for decoding an encoded block of an encoded video frame, the apparatus comprising:

a processor configured to execute instructions stored in a non-transitory memory to;

determine, based on one or more syntax elements encoded to a bitstream including the encoded video frame, whether the encoded block was encoded by weighting distances from each of a plurality of reference frames to the encoded video frame;

responsive to a determination that the encoded block was encoded by weighting the distances from each of the plurality of reference frames to the encoded video frame, determine a compound motion block by applying a first quantized weighting coefficient to pixel values from a video block of a first reference frame of the plurality of reference frames and by applying a second quantized weighting coefficient to pixel values from a video block of a second reference frame of the plurality of reference frames, the first quantized weighting coefficient and the second quantized weighting coefficient determined based on a ratio of a first distance and a second distance, the first distance representing a distance from the first reference frame to the encoded video frame, the second distance representing a distance from the second reference frame to the encoded video frame;

generate a prediction block using the compound motion block; and

decode the encoded block using the prediction block.

11. The apparatus of claim 10, wherein,

when the first distance is greater than the second distance, determine that the first quantized weighting coefficient has a value of 1 and the second quantized weighting coefficient has a value of at least 1,

and, when the first distance is less than the second distance, the first quantized weighting coefficient has a value of at least 1 and the second quantized weighting coefficient has a value of 1.

12. The apparatus of claim 10, wherein the instructions to determine the compound motion block include instructions to:

determine a first value by applying the first quantized weighting coefficient against pixel values from the video block of the first reference frame; and

determine a second value by applying the second quantized weighting coefficient against pixel values from the video block of the second reference frame,

wherein the compound motion block is determined by dividing a sum of the first value and the second value by a sum of the first quantized weighting coefficient and the second quantized weighting coefficient.

13. The apparatus of claim 10, wherein the compound motion block is a first compound motion block and the prediction block is a first prediction block, wherein the instructions include instructions to:

responsive to a determination that the encoded block was not encoded by weighting the distances using the one or more quantized weighting coefficients, determine a second compound motion block based on an average of pixel values from an encoded video block of the first reference frame and pixel values from an encoded video block of the second reference frame;

generate a second prediction block using the second compound motion block; and

decode the encoded block using the second prediction block.

14. The apparatus of claim 10, wherein the instructions include instructions to:

decode the one or more syntax elements from the bitstream prior to the determination as to whether the encoded block was encoded by weighting the distances from each of the plurality of reference frames to the encoded video frame.

15. The apparatus of claim 10, wherein an order of encoded video frames of a video sequence including the encoded video frame, the first reference frame, and the second reference frame is indicated within the bitstream.

20. The apparatus of claim 10, wherein, when the first distance is greater than the second distance, the ratio is a ratio of the first distance over the second distance, and, when the first distance is not greater than the second distance, the ratio is a ratio of the second distance over the first distance.