SUB-PREDICTION UNIT (PU) BASED TEMPORAL MOTION VECTOR PREDICTION IN HEVC AND SUB-PU DESIGN IN 3D-HEVC

• US 20150085935A1
• Filed: 09/25/2014
• Published: 03/26/2015
• Est. Priority Date: 09/26/2013
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Chen, Ying, Van der Auwera, Geert, Zhang, Li, Liu, Hongbin
• Granted Patent: US 9,667,996 B2
• Time in Patent Office: Days
• US Class Current: 375/240.16
• CPC Class Codes: H04N 19/197 including determination of the initial value of an encoding parameter H04N19/56 takes precedence
• CPC Class Codes: H04N 19/52 by predictive encoding
• CPC Class Codes: H04N 19/53 Multi-resolution motion estimation; Hierarchical motion estimation
• CPC Class Codes: H04N 19/56 Motion estimation with initialisation of the vector search, e.g. estimating a good candidate to initiate a search
• CPC Class Codes: H04N 19/597 specially adapted for multi-view video sequence encoding

Abstract

Techniques are described for sub-prediction unit (PU) based motion prediction for video coding in HEVC and 3D-HEVC. In one example, the techniques include an advanced temporal motion vector prediction (TMVP) mode to predict sub-PUs of a PU in single layer coding for which motion vector refinement may be allowed. The advanced TMVP mode includes determining motion vectors for the PU in at least two stages to derive motion information for the PU that includes different motion vectors and reference indices for each of the sub-PUs of the PU. In another example, the techniques include storing separate motion information derived for each sub-PU of a current PU predicted using a sub-PU backward view synthesis prediction (BVSP) mode even after motion compensation is performed. The additional motion information stored for the current PU may be used to predict subsequent PUs for which the current PU is a neighboring block.

Claims

1. A method of processing video data, the method comprising:

determining first motion information for a prediction unit (PU) of a coding unit (CU) of a video block from neighboring blocks of the PU according to a backward view synthesis prediction (BVSP) mode, wherein the first motion information includes at least one disparity motion vector and an associated reference view index that identifies an inter-view reference picture;

partitioning the PU into two or more sub-PUs;

determining second motion information for each of the sub-PUs, wherein the second motion information includes at least one disparity motion vector associated with a depth block of the inter-view reference picture corresponding to each of the sub-PUs;

performing motion compensation to predict each of the sub-PUs from the inter-view reference picture based on the second motion information; and

storing the second motion information for each of the sub-PUs of the PU in a memory to be used for predicting subsequent PUs.

2. The method of claim 1, further comprising generating a merge mode candidate list of motion information for predicting a subsequent PU, wherein the PU is a neighboring block of the subsequent PU and generating the merge mode candidate list comprises including the second motion information for at least one of the sub-PUs of the PU stored in the memory as a sub-PU BVSP candidate in the merge mode candidate list.

3. The method of claim 1, wherein determining the first motion information for the PU comprises selecting the first motion information as a BVSP candidate from a merge mode candidate list of motion information for predicting the PU.

4. The method of claim 1, where determining the second motion information for each of the sub-PUs comprises deriving the at least one disparity motion vector for each of the sub-PUs to have a horizontal component equal to a maximum value of four corner pixels for the depth block of the inter-view reference picture corresponding to each of the sub-PUs, and a vertical component equal to zero.

5. The method of claim 1, wherein determining the second motion information for each of the sub-PUs comprises selecting the second motion information as a sub-PU BVSP candidate from a merge mode candidate list of motion information for predicting sub-PUs.

6. The method of claim 1, wherein the memory comprises a decoded picture buffer, and wherein storing the second motion information comprises storing the disparity motion vector for each of the sub-PUs in the decoded picture buffer with a reference picture list that includes the inter-view reference picture identified by the reference view index of the first motion information for the PU.

7. The method of claim 1, further comprising:

generating a predictive block for each of the sub-PUs of the PU from the inter-view reference picture based on the second motion information;

generating a residual block based on the video block and the predictive block for each of the sub-PUs; and

encoding the residual block and an indicator of at least the first motion information for the PU in a video bitstream.

8. The method of claim 1, further comprising:

decoding a residual block and an indicator of at least the first motion information for the PU from a received video bitstream;

generating a predictive block for each of the sub-PUs of the PU from the inter-view reference picture based on the second motion information; and

generating a reconstructed version of the video block based on the residual block and the predictive block for each of the sub-PUs.

9. The method of claim 1, further comprising applying a deblocking filter to each CU of a reconstructed version of the video block prior to storing the video block in a decoded picture buffer, wherein applying the deblocking filter to the CU of the video block that includes the PU with the two or more sub-PUs comprises:

prior to applying the deblocking filter to the CU, creating artificial PU boundaries or artificial transform unit (TU) boundaries at sub-PU boundaries between two adjacent sub-PUs in the PU; and

applying the deblocking filter to at least one of PU boundaries between two adjacent PUs of the CU or TU boundaries between two adjacent TUs of the CU, including the artificial PU boundaries or the artificial TU boundaries.

10. The method of claim 9, wherein creating the artificial PU boundaries or the artificial TU boundaries comprises converting a transform tree of the CU in order to associate the PU with a transform tree hierarchy and associate each of the sub-PUs with a TU, wherein the sub-PU boundaries comprise artificial TU boundaries.

11. The method of claim 9, wherein creating the artificial PU boundaries or the artificial TU boundaries comprises converting a transform tree of the CU in order to associate the PU with a transform tree hierarchy and associate at least a part of each of the sub-PUs with a TU, wherein the sub-PU boundaries comprise artificial TU

12. The method of claim 9, wherein creating the artificial PU boundaries or the artificial TU boundaries comprises converting the CU to a coding tree in order to associate the PU with a CU and associate each of the sub-PUs with a PU, wherein the sub-PU boundaries comprise artificial PU boundaries.

13. A video processing device comprising:

a memory configured to store video data; and

one or more processors in communication with the memory and configured to;

determine first motion information for a prediction unit (PU) of a coding unit (CU) of a video block from neighboring blocks of the PU according to a backward view synthesis prediction (BVSP) mode, wherein the first motion information includes at least one disparity motion vector and an associated reference view index that identifies an inter-view reference picture,

partition the PU into two or more sub-PUs,

determine second motion information for each of the sub-PUs, wherein the second motion information includes at least one disparity motion vector associated with a depth block of the inter-view reference picture corresponding to each of the sub-PUs, and

perform motion compensation to predict each of the sub-PUs from the inter-view reference picture based on the second motion information,

wherein the memory is configured to store the second motion information for each of the sub-PUs of the PU to be used for predicting subsequent PUs.

14. The device of claim 13, wherein the one or more processors are configured to generate a merge mode candidate list of motion information for predicting a subsequent PU, wherein the PU is a neighboring block of the subsequent PU and the merge mode candidate list includes the second motion information for at least one of the sub-PUs of the PU stored in the memory as a sub-PU BVSP candidate in the merge mode candidate list.

15. The device of claim 13, wherein the one or more processors are configured to select the first motion information as a BVSP candidate from a merge mode candidate list of motion information for predicting the PU.

16. The device of claim 13, where the one or more processors are configured to derive the at least one disparity motion vector included in the second motion information for each of the sub-PUs to have a horizontal component equal to a maximum value of four corner pixels for the depth block of the inter-view reference picture corresponding to each of the sub-PUs, and a vertical component equal to zero.

17. The device of claim 13, wherein the one or more processors are configured to select the second motion information for each of the sub-PUs as a sub-PU BVSP candidate from a merge mode candidate list of motion information for predicting sub-PUs.

18. The device of claim 13, wherein the memory comprises a decoded picture buffer, and wherein the one or more processors are configured to store the disparity motion vector included in the second motion information for each of the sub-PUs in the decoded picture buffer with a reference picture list that includes the inter-view reference picture identified by the reference view index of the first motion information for the PU.

19. The device of claim 13, wherein the video processing device comprises a video encoding device, and wherein the one or more processors are configured to:

generate a predictive block for each of the sub-PUs of the PU from the inter-view reference picture based on the second motion information;

generate a residual block based on the video block and the predictive block for each of the sub-PUs; and

encode the residual block and an indicator of at least the first motion information for the PU in a video bitstream.

20. The device of claim 13, wherein the video processing device comprises a video decoding device, and wherein the one or more processors are configured to:

decode a residual block and an indicator of at least the first motion information for the PU from a received video bitstream;

generate a predictive block for each of the sub-PUs of the PU from the inter-view reference picture based on the second motion information; and

generate a reconstructed version of the video block based on the residual block and the predictive block for each of the sub-PUs.

21. The device of claim 13, wherein the one or more processors are configured to apply a deblocking filter to each CU of a reconstructed version of the video block prior to storing the video block in a decoded picture buffer, and wherein, to apply the deblocking filter to the CU of the video block that includes the PU with the two or more sub-PUs, the one or more processors are configured to:

prior to applying the deblocking filter to the CU, create artificial PU boundaries or artificial transform unit (TU) boundaries at sub-PU boundaries between two adjacent sub-PUs in the PU; and

apply the deblocking filter to at least one of PU boundaries between two adjacent PUs of the CU or TU boundaries between two adjacent TUs of the CU, including the artificial PU boundaries or the artificial TU boundaries.

22. The device of claim 21, wherein the one or more processors are configured to convert a transform tree of the CU in order to associate the PU with a transform tree hierarchy and associate each of the sub-PUs with a TU, wherein the sub-PU boundaries comprise artificial TU boundaries.

23. The device of claim 21, wherein the one or more processors are configured to convert a transform tree of the CU in order to associate the PU with a transform tree hierarchy and associate at least a part of each of the sub-PUs with a TU, wherein the sub-PU boundaries comprise artificial TU

24. The device of claim 21, wherein the one or more processors are configured to convert the CU to a coding tree in order to associate the PU with a CU and associate each of the sub-PUs with a PU, wherein the sub-PU boundaries comprise artificial PU boundaries.

25. A video processing device comprising:

means for determining first motion information for a prediction unit (PU) of a coding unit (CU) of a video block from neighboring blocks of the PU according to a backward view synthesis prediction (BVSP) mode, wherein the first motion information includes at least one disparity motion vector and an associated reference view index that identifies an inter-view reference picture;

means for partitioning the PU into two or more sub-PUs;

means for determining second motion information for each of the sub-PUs, wherein the second motion information includes at least one disparity motion vector associated with a depth block of the inter-view reference picture corresponding to each of the sub-PUs;

means for performing motion compensation to predict each of the sub-PUs from the inter-view reference picture based on the second motion information; and

means for storing the second motion information for each of the sub-PUs of the PU in a memory to be used for predicting subsequent PUs.

26. A computer-readable storage medium storing instructions for processing video data that, when executed, cause one or more processors to:

determine first motion information for a prediction unit (PU) of a coding unit (CU) of a video block from neighboring blocks of the PU according to a backward view synthesis prediction (BVSP) mode, wherein the first motion information includes at least one disparity motion vector and an associated reference view index that identifies an inter-view reference picture;

partition the PU into two or more sub-PUs;

determine second motion information for each of the sub-PUs, wherein the second motion information includes at least one disparity motion vector associated with a depth block of the inter-view reference picture corresponding to each of the sub-PUs;

perform motion compensation to predict each of the sub-PUs from the inter-view reference picture based on the second motion information; and

store the second motion information for each of the sub-PUs of the PU in a memory to be used for predicting subsequent PUs.