Constrained block-level optimization and signaling for video coding tools

• US 10,623,774 B2
• Filed: 03/21/2017
• Issued: 04/14/2020
• Est. Priority Date: 03/22/2016
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Zhao, Xin, Seregin, Vadim, Joshi, Rajan Laxman, Said, Amir, Li, Xiang, Karczewicz, Marta, Chen, Jianle, Chien, Wei-Jung
• Primary Examiner(s): Vaughn, Jr., William C
• Assistant Examiner(s): Belai, Naod W
• Application Number:
  US15/465,122
• Publication Number:
  US 20170280162A1
• Time in Patent Office: 1,120 Days
• Field of Search: 37524012
• CPC Class Codes: H04N 19/103 Selection of coding mode or of prediction mode
• CPC Class Codes: H04N 19/119 Adaptive subdivision aspects, e.g. subdivision of a picture into rectangular or non-rectangular coding blocks
• CPC Class Codes: H04N 19/122 Selection of transform size, e.g. 8x8 or 2x4x8 DCT; Selection of sub-band transforms of varying structure or type
• CPC Class Codes: H04N 19/134 characterised by the element, parameter or criterion affecting or controlling the adaptive coding
• CPC Class Codes: H04N 19/159 Prediction type, e.g. intra-frame, inter-frame or bidirectional frame prediction
• CPC Class Codes: H04N 19/176 the region being a block, e.g. a macroblock
• CPC Class Codes: H04N 19/184 the unit being bits, e.g. of the compressed video stream
• CPC Class Codes: H04N 19/46 Embedding additional information in the video signal during the compression process H04N19/517, H04N19/68, H04N19/70 take precedence
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 19/70 characterised by syntax aspects related to video coding, e.g. related to compression standards
• CPC Class Codes: H04N 19/96 Tree coding, e.g. quad-tree coding

Abstract

An example device for decoding encoded video data includes storage media and processing circuitry. The storage media are configured a portion of the encoded video data. The processing circuitry is configured to determine a block-level threshold for the portion of the encoded video data stored to the storage media, to determine that an encoded block of the portion of the encoded video data has a size that is equal to or greater than the threshold, to receive a syntax element indicating that a portion of the encoded block is to be reconstructed using a coding tool, to determine, based on the encoded block having the size that is equal to or greater than the threshold, that the syntax element applies to all samples of a plurality of samples included in the encoded block, and to reconstruct the encoded block based on the coding tool.

Claims

1. A device for decoding encoded video data, the device comprising:

storage media configured to store at least a portion of the encoded video data; and

processing circuitry in communication with the storage media, the processing circuitry being configured to;

determine a block-level threshold for the portion of the encoded video data stored to the storage media;

determine that an encoded block of the portion of the encoded video data stored to the storage media has a size that is equal to or greater than the block-level threshold;

determine that the encoded block is partitioned into a plurality of sub-blocks according to one of a quadtree-based partitioning scheme or a quadtree plus binary tree-based partitioning scheme;

receive a syntax element indicating that a first sub-block of the plurality of sub-blocks of the encoded block is to be reconstructed using a coding tool;

based on the encoded block having the size that is equal to or greater than the block-level threshold, determine that the syntax element applies to all samples of a plurality of samples included in all sub-blocks of the plurality of sub-blocks of the encoded block; and

reconstruct all samples of the plurality of samples included in all sub-blocks of the plurality of sub-blocks of the encoded block using the coding tool.

2. The device of claim 1, wherein the block-level threshold represents a minimum area.

3. The device of claim 1, wherein the received syntax element comprises an enhanced multiple transform (EMT) flag, the processing circuitry being further configured to:

determine that the EMT flag is set to an enabled state; and

based on the EMT flag being set to the enabled state, decode a respective EMT index for each respective sub-block of the plurality of sub-blocks of the encoded block,

wherein to reconstruct all samples of the plurality of samples included in all sub-blocks of the plurality of sub-blocks of the encoded block, the processing circuitry is configured to reconstruct all samples of the plurality of samples included in all sub-blocks of the plurality of sub-blocks of the encoded block using an EMT coding tool.

4. The device of claim 1, the block-level threshold representing a maximum sub-block area, the coding tool comprising at least one of intra prediction or inter coding, and the processing circuitry being further configured to:

determine, based on the first sub-block of the plurality of sub-blocks being smaller than the maximum sub-block area, that the first sub-block is to be reconstructed using only one of intra prediction or inter prediction.

5. The device of claim 1, the processing circuitry being further configured to apply the block-level threshold to each respective coding tool of a plurality of coding tools that includes the coding tool for which the syntax element is received.

6. The device of claim 5, wherein the plurality of coding tools includes two or more of an enhanced multiple transform (EMT) coding tool, a position-dependent prediction combination (PDPC) coding tool, a non-separable secondary transform (NSST) coding tool, an intra prediction tool, or an inter prediction tool.

7. The device of claim 1, wherein the block-level threshold is included in a plurality of block-level thresholds, and wherein the processing circuitry is further configured to apply a respective block-level threshold of the plurality of block-level thresholds to each respective coding tool of a plurality of coding tools that includes the coding tool for which the syntax element is received.

8. The device of claim 1, wherein the encoded block comprises a real block of the portion of the encoded video data stored to the storage media, the processing circuitry being further configured to:

receive the syntax element with respect to the real block instead of receiving the syntax element with respect to a virtual block that corresponds to the real block.

9. The device of claim 1, wherein to determine the block-level threshold, the processing circuitry is configured to receive data indicating the block-level threshold in one of a picture parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS) of an encoded video bitstream.

10. A device for encoding video data, the method comprising:

storage media configured to store at least a portion of the video data; and

processing circuitry in communication with the storage media, the processing circuitry being configured to;

determine a block-level threshold for the portion of the video data stored to the storage media;

determine that a block of the portion of the video data stored to the storage media has a size that is equal to or greater than the block-level threshold;

partition the encoded block into a plurality of sub-blocks according to one of a quadtree-based partitioning scheme or a quadtree plus binary tree-based partitioning scheme;

encode all samples of a first sub-block of the plurality of sub-blocks partitioned from the block using a coding tool;

based on the encoded block having the size that is equal to or greater than the block-level threshold, encode all samples of a plurality of samples included in all sub-blocks of the plurality of sub-blocks partitioned from the block using the coding tool, to form an encoded block; and

based on the encoded block having the size that is equal to or greater than the block-level threshold, encode a syntax element indicating that all samples of the plurality of samples included in all sub-blocks of the plurality of sub-blocks partitioned from the encoded block are to be reconstructed using the coding tool.

11. The device of claim 10, wherein the block-level threshold represents a minimum sub-block area.

12. The device of claim 10, wherein the encoded syntax element comprises an enhanced multiple transform (EMT) flag, the processing circuitry being further configured to:

set the EMT flag to an enabled state; and

encode a respective EMT index for each respective sub-block of the plurality of sub-blocks of the encoded block.

13. The device of claim 10, the block-level threshold representing a maximum sub-block area, the coding tool comprising at least one of intra coding or inter coding, and the processing circuitry being further configured to:

based on the first sub-block of the one or more sub-blocks being smaller than the maximum sub-block area, encode the first sub-block using only one of intra coding or inter coding.

14. The device of claim 10, the processing circuitry being further configured to apply the block-level threshold to each respective coding tool of a plurality of coding tools that includes the coding tool for which the syntax element is encoded.

15. The device of claim 14, wherein the plurality of coding tools includes two or more of an enhanced multiple transform (EMT) coding tool, a position-dependent prediction combination (PDPC) coding tool, a non-separable secondary transform (NSST) coding tool, an intra prediction tool, or an inter prediction tool.

16. The device of claim 10, wherein the block-level threshold is included in a plurality of block-level thresholds, and wherein the processing circuitry is further configured to apply a respective block-level threshold of the plurality of block-level thresholds to each respective coding tool of a plurality of coding tools that includes the coding tool for which the syntax element is encoded.

17. The device of claim 10, wherein the encoded block comprises a real block of the portion of the video data stored to the storage media, the processing circuitry being further configured to:

determine that a virtual block that corresponds to the real block meets the block-level threshold; and

based on the virtual block meeting the block-level threshold, signal the encoded syntax element with respect to the real block instead of signaling the encoded syntax element with respect to the virtual block that corresponds to the real block.

18. The device of claim 10, the processing circuitry being further configured to signal data indicative of the block-level threshold in one of a picture parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS) of an encoded video bitstream.

19. The device of claim 10, the processing circuitry being further configured to determine the block-level threshold based on one or more of an area of the block, a width of the block, or a height of the block.

20. The device of claim 10, the processing circuitry being further configured to determine whether to apply the coding tool in encoding the block at least in part by performing a bitrate optimization with respect to the coding tool and the first sub-block.

21. The device of claim 10, the processing circuitry being further configured to omit, from an encoded video bitstream to be signaled via an interface of the device, syntax elements indicating the coding tool used to reconstruct individual samples of the encoded block.

22. A method of decoding encoded video data, the method comprising:

determining a block-level threshold for a portion of the encoded video data;

determining that an encoded block of the portion of the encoded video data has a size that is equal to or greater than the block-level threshold;

determining that the encoded block is partitioned into a plurality of sub-blocks according to one of a quadtree-based partitioning scheme or a quadtree plus binary tree-based partitioning scheme;

receiving a syntax element indicating that a first sub-block of the plurality of sub-blocks of the encoded block is to be reconstructed using a coding tool;

based on the encoded block having the size that is equal to or greater than the block-level threshold, determining that the syntax element applies to all samples of a plurality of samples included in all sub-blocks of the plurality of sub-blocks of the encoded block; and

reconstructing all samples of the plurality of samples included in all sub-blocks of the plurality of sub-blocks of the encoded block using the coding tool.

23. The method of claim 22, wherein the block-level threshold represents a minimum area.

24. The method of claim 22, wherein the received syntax element comprises an enhanced multiple transform (EMT) flag, the method further comprising:

determining that the EMT flag is set to an enabled state; and

based on the EMT flag being set to the enabled state, decoding a respective EMT index for each respective sub-block of the plurality of sub-blocks of the encoded block,

wherein reconstructing all samples of the plurality of samples included in all sub-blocks of the plurality of sub-blocks of the encoded block comprises reconstructing all samples of the plurality of samples included in all sub-blocks of the plurality of sub-blocks of using an EMT coding tool.

25. The method of claim 22, the block-level threshold representing a maximum sub-block area, the coding tool comprising at least one of intra prediction or inter prediction, the method further comprising:

determining, based on the first sub-block of the plurality of sub-blocks being smaller than the maximum sub-block area, that the first sub-block is to be reconstructed using only one of intra prediction or inter prediction.

26. The method of claim 22, further comprising applying the block-level threshold to each respective coding tool of a plurality of coding tools that includes the coding tool for which the syntax element is received.

27. The method of claim 26, wherein the plurality of coding tools includes two or more of an enhanced multiple transform (EMT) coding tool, a position-dependent prediction combination (PDPC) coding tool, a non-separable secondary transform (NSST) coding tool, an intra prediction tool, or an inter prediction tool.

28. The method of claim 22, the block-level threshold being included in a plurality of block-level thresholds, the method further comprising applying a respective block-level threshold of the plurality of block-level thresholds to each respective coding tool of a plurality of coding tools that includes the coding tool for which the syntax element is received.

29. The method of claim 22, wherein the encoded block comprises a real block of the portion of the encoded video data stored to the storage media, the method further comprising:

receiving the syntax element with respect to the real block instead of receiving the syntax element with respect to a virtual block that corresponds to the real block.

30. An apparatus for decoding encoded video data, the apparatus comprising:

means for determining a block-level threshold for a portion of the encoded video data;

means for determining that an encoded block of the portion of the encoded video data stored to the storage media has a size that is equal to or greater than the block-level threshold;

means for determining that the encoded block is partitioned into a plurality of sub-blocks according to one of a quadtree-based partitioning scheme or a quadtree plus binary tree-based partitioning scheme;

means for receiving a syntax element indicating that a first sub-block of the plurality of sub-blocks of encoded block is to be reconstructed using a coding tool;

means for determining, based on the encoded block having the size that is equal to or greater than the block-level threshold, that the syntax element applies to all samples of a plurality of samples included in all sub-blocks of the plurality of sub-blocks of the encoded block; and

means for reconstructing all samples of the plurality of samples included in all sub-blocks of the plurality of sub-blocks of the encoded block using the coding tool.