High-level syntax extensions for high efficiency video coding

• US 10,200,709 B2
• Filed: 03/13/2013
• Issued: 02/05/2019
• Est. Priority Date: 03/16/2012
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Chen, Ying, Wang, Ye-Kui, Zhang, Li
• Primary Examiner(s): Pontius, James M
• Assistant Examiner(s): Lotfi, Kyle M
• Application Number:
  US13/801,731
• Publication Number:
  US 20130243081A1
• Time in Patent Office: 2,155 Days
• CPC Class Codes: H04N 19/51 Motion estimation or motion compensation
• CPC Class Codes: H04N 19/52 by predictive encoding
• CPC Class Codes: H04N 19/58 Motion compensation with long-term prediction, i.e. the reference frame for a current frame not being the temporally closest one H04N19/23 takes precedence
• CPC Class Codes: H04N 19/597 specially adapted for multi-view video sequence encoding
• CPC Class Codes: H04N 19/70 characterised by syntax aspects related to video coding, e.g. related to compression standards

Abstract

In one example, a device includes a video coder configured to code a picture order count (POC) value for a first picture of video data, code a second-dimension picture identifier for the first picture, and code, in accordance with a base video coding specification or an extension to the base video coding specification, a second picture based at least in part on the POC value and the second-dimension picture identifier of the first picture. The video coder may comprise a video encoder or a video decoder. The second-dimension picture identifier may comprise, for example, a view identifier, a view order index, a layer identifier, or other such identifier. The video coder may code the POC value and the second-dimension picture identifier during coding of a motion vector for a block of the second picture, e.g., during advanced motion vector prediction or merge mode coding.

Claims

1. A method of decoding video data, the method comprising:

determining that a first motion vector of a first block of a picture refers to a short-term reference picture;

determining that a second motion vector of a second block of the picture refers to a long-term reference picture; and

based on the determination that the first motion vector refers to the short-term reference picture and the determination that the second motion vector refers to the long-term reference picture, disabling motion vector prediction between the first motion vector of the first block of the picture and the second motion vector of the second block of the picture, wherein the first block and the second block are spatial neighbors in the picture, and wherein disabling motion vector prediction between the first motion vector and the second motion vector comprises;

at least one of processing a value for a flag indicating whether the second motion vector is available for use as a first motion vector predictor for the first motion value equal to a value indicating that the second motion vector is not available for use as the first motion vector predictor for the first motion vector or processing a value for a flag indicating whether the first motion vector is available for use as a second motion vector predictor for the second motion value equal to a value indicating that the first motion vector is not available for use as the second motion vector predictor for the second motion vector; and

decoding the first motion vector without using the second motion vector to predict the first motion vector and decoding the second motion vector without using the first motion vector to predict the second motion vector.

2. The method of claim 1, further comprising:

enabling prediction between a first short-term motion vector of the picture and a second short-term motion vector of the picture; and

scaling at least one of the first short-term motion vector and the second short-term motion vector based on a picture order count (POC) value for a first short-term reference picture referred to by the first short-term motion vector and a POC value for a second short-term reference picture referred to by the second short-term motion vector.

3. The method of claim 1, further comprising decoding a value indicating whether a second picture of the video data comprises a long-term reference picture, wherein the value indicating whether the second picture comprises the long-term reference picture further indicates whether the second picture is used for inter-view prediction.

4. The method of claim 1,

wherein determining that the first motion vector of the first block of the second picture refers to the short-term reference picture comprises determining that the first motion vector refers to a first reference picture marked as used for short-term reference, and

wherein determining that the second motion vector of the second block of the second picture refers to the long-term reference picture comprises determining that the second motion vector refers to a second reference picture marked as used for long-term reference.

5. The method of claim 1, wherein the picture comprises a second picture, the method further comprising:

decoding data of the second picture that refers to a picture order count (POC) value for a first picture of video data;

decoding data of the second picture that refers to a second-dimension picture identifier for the first picture; and

decoding, in accordance with a base video coding specification, the second picture based at least in part on the POC value and the second-dimension picture identifier of the first picture.

6. The method of claim 5, wherein decoding the second picture comprises:

identifying the first picture using the POC value and the second-dimension picture identifier; and

decoding at least a portion of the second picture relative to the first picture.

7. The method of claim 6, wherein identifying the first picture comprises identifying the first picture during decoding of a motion vector for a block of the second picture, wherein decoding of the motion vector comprises decoding the motion vector according to at least one of advanced motion vector prediction (AMVP), temporal motion vector prediction (TMVP), and merge mode.

8. The method of claim 5, further comprising decoding, in accordance with an extension to the base video coding specification, a third picture based at least in part on the POC value and the second-dimension picture identifier of the first picture.

9. The method of claim 8, further comprising, prior to decoding the third picture, marking all inter-view reference pictures, including the first picture, as long-term reference pictures.

10. The method of claim 9, further comprising:

storing a status for each of the inter-view reference pictures for the third picture, wherein the status comprises one of long-term reference picture, short-term reference picture, and unused for reference, prior to marking the inter-view reference pictures as long-term reference pictures, wherein the inter-view reference pictures include the first picture; and

after decoding the second picture, setting new statuses for each of the inter-view reference pictures based on the stored statuses.

11. The method of claim 8, wherein the base video coding specification comprises High Efficiency Video Coding (HEVC) base specification, and wherein the extension to the base video coding specification comprises one of a Scalable Video Coding (SVC) extension to the HEVC base specification and a Multiview Video Coding (MVC) extension to the HEVC base specification.

12. The method of claim 8, wherein the second-dimension picture identifier comprises at least one of a view identifier for a view including the first picture, a view order index for the view including the first picture, a combination of the view order index and a depth flag, a layer identifier for a scalable video coding (SVC) layer including the first picture, and a generic layer identifier.

13. The method of claim 8, further comprising, after decoding the third picture, marking each inter-view reference picture as one of a long-term reference picture, a short-term reference picture, and unused for reference.

14. The method of claim 13, further comprising:

after marking an inter-view reference picture as a long-term reference picture, assigning the inter-view reference picture a new POC value that is currently unused; and

after decoding the second picture, restoring an original POC value for the inter-view reference picture.

15. The method of claim 14, wherein the original POC value comprises the POC value of the first picture.

16. A method of encoding video data, the method comprising:

determining that a first motion vector of a first block of a picture refers to a short-term reference picture;

determining that a second motion vector of a second block of the picture refers to a long-term reference picture; and

based on the determination that the first motion vector refers to the short-term reference picture and the determination that the second motion vector refers to the long-term reference picture, disabling motion vector prediction between the first motion vector of the first block of the picture and the second motion vector of the second block of the picture, wherein the first block and the second block are spatial neighbors in the picture, and wherein disabling motion vector prediction between the first motion vector and the second motion vector comprises;

at least one of processing a value for a flag indicating whether the second motion vector is available for use as a first motion vector predictor for the first motion value equal to a value indicating that the second motion vector is not available for use as the first motion vector predictor for the first motion vector or processing a value for a flag indicating whether the first motion vector is available for use as a second motion vector predictor for the second motion value equal to a value indicating that the first motion vector is not available for use as the second motion vector predictor for the second motion vector; and

encoding the first motion vector without using the second motion vector to predict the first motion vector and encoding the second motion vector without using the first motion vector to predict the second motion vector.

17. The method of claim 16, further comprising:

enabling prediction between a first short-term motion vector of the picture and a second short-term motion vector of the picture; and

scaling at least one of the first short-term motion vector and the second short-term motion vector based on a picture order count (POC) value for a first short-term reference picture referred to by the first short-term motion vector and a POC value for a second short-term reference picture referred to by the second short-term motion vector.

18. The method of claim 16, wherein the second-dimension picture identifier comprises at least one of a view identifier for a view including the first picture, a view order index for the view including the first picture, a combination of the view order index and a depth flag, a layer identifier for a scalable video coding (SVC) layer including the first picture, and a generic layer identifier.

19. The method of claim 16, further comprising encoding a value indicating whether a second picture of the video data comprises a long-term reference picture, wherein the value indicating whether the second picture comprises the long-term reference picture further indicates whether the second picture is used for inter-view prediction.

20. The method of claim 16,

wherein determining that the first motion vector of the first block of the second picture refers to the short-term reference picture comprises determining that the first motion vector refers to a first reference picture marked as used for short-term reference, and

wherein determining that the second motion vector of the second block of the second picture refers to the long-term reference picture comprises determining that the second motion vector refers to a second reference picture marked as used for long-term reference.

21. The method of claim 16, wherein the picture comprises a second picture, the method further comprising:

encoding data of a second picture that refers to a picture order count (POC) value for a first picture of video data;

encoding data of the second picture that refers to a second-dimension picture identifier for the first picture; and

encoding, in accordance with a base video coding specification, the second picture based at least in part on the POC value and the second-dimension picture identifier of the first picture.

22. The method of claim 21, further comprising:

identifying the first picture using the POC value and the second-dimension picture identifier; and

coding at least a portion of the second picture relative to the first picture.

23. The method of claim 22, wherein identifying the first picture comprises identifying the first picture during encoding of the motion vector for a block of the second picture, wherein encoding of the motion vector comprises encoding the motion vector according to at least one of advanced motion vector prediction (AMVP), temporal motion vector prediction (TMVP), and merge mode.

24. The method of claim 21, further comprising encoding, in accordance with an extension to the base video coding specification, a third picture based at least in part on the POC value and the second-dimension picture identifier of the first picture.

25. The method of claim 24, further comprising, prior to encoding the third picture, marking all inter-view reference pictures as long-term reference pictures.

26. The method of claim 25, further comprising:

storing a status for each of the inter-view reference pictures, wherein the status comprises one of long-term reference picture, short-term reference picture, and unused for reference, prior to marking the inter-view reference pictures as long-term reference pictures; and

after encoding the second picture, setting new statuses for each of the inter-view reference pictures based on the stored statuses.

27. The method of claim 24, further comprising, after encoding the third picture, marking each inter-view reference picture as one of a long-term reference picture, a short-term reference picture, and unused for reference.

28. The method of claim 27, further comprising:

after marking an inter-view reference picture as a long-term reference picture, assigning the inter-view reference picture a new POC value that is currently unused; and

after encoding the second picture, restoring an original POC value for the inter-view reference picture.

29. The method of claim 28, wherein the original POC value comprises the POC value of the second picture.

30. The method of claim 24, wherein the base video coding specification comprises High Efficiency Video Coding (HEVC) base specification, and wherein the extension to the base video coding specification comprises one of a Scalable Video Coding (SVC) extension to the HEVC base specification and a Multiview Video Coding (MVC) extension to the HEVC base specification.

31. A device for decoding video data, the device comprising:

a memory configured to store video data; and

a video decoder configured to;

determine that a first motion vector of a first block of a picture refers to a short-term reference picture,

determine that a second motion vector of a second block of the picture refers to a long-term reference picture, and

based on the determination that the first motion vector refers to the short-term reference picture and the determination that the second motion vector refers to the long-term reference picture, disable motion vector prediction between the first motion vector of the first block of the picture and the second motion vector of the second block of the picture, wherein the first block and the second block are spatial neighbors in the picture, and wherein to disable motion vector prediction between the first motion vector and the second motion vector, the video decoder is configured to;

at least one of process a value for a flag indicating whether the second motion vector is available for use as a first motion vector predictor for the first motion value equal to a value indicating that the second motion vector is not available for use as the first motion vector predictor for the first motion vector or process a value for a flag indicating whether the first motion vector is available for use as a second motion vector predictor for the second motion value equal to a value indicating that the first motion vector is not available for use as the second motion vector predictor for the second motion vector; and

decode the first motion vector without using the second motion vector to predict the first motion vector and decode the second motion vector without using the first motion vector to predict the second motion vector.

32. The device of claim 31, wherein the device comprises at least one of:

an integrated circuit;

a microprocessor; and

a wireless communication device that includes the video decoder.

33. The device of claim 31, wherein the video decoder is configured to enable prediction between a first short-term motion vector of the picture and a second short-term motion vector of the picture, and scale at least one of the first short-term motion vector and the second short-term motion vector based on a picture order count (POC) value for a first short-term reference picture referred to by the first short-term motion vector and a POC value for a second short-term reference picture referred to by the second short-term motion vector.

34. The device of claim 31, wherein the second-dimension picture identifier comprises at least one of a view identifier for a view including the first picture, a view order index for the view including the first picture, a combination of the view order index and a depth flag, a layer identifier for a scalable video coding (SVC) layer including the first picture, and a generic layer identifier.

35. The device of claim 31, wherein the video decoder is configured to decode a value indicating whether a second picture of the video data comprises a long-term reference picture, wherein the value indicating whether the second picture comprises the long-term reference picture further indicates whether the second picture is used for inter-view prediction.

36. The device of claim 31,

wherein to determine that the first motion vector of the first block of the second picture refers to the short-term reference picture, the video decoder is configured to determine that the first motion vector refers to a first reference picture marked as used for short-term reference, and

wherein to determine that the second motion vector of the second block of the second picture refers to the long-term reference picture, the video decoder is configured to determine that the second motion vector refers to a second reference picture marked as used for long-term reference.

37. The device of claim 31, wherein the picture comprises a second picture, and wherein the video decoder is further configured to:

decode data of the second picture that refers to a picture order count (POC) value for a first picture of the video data,

decode data of the second picture that refers to a second-dimension picture identifier for the first picture, and

decode, in accordance with a base video coding specification, the second picture based at least in part on the POC value and the second-dimension picture identifier of the first picture.

38. The device of claim 37, wherein the video decoder is configured to identify the first picture using the POC value and the second-dimension picture identifier, and decode at least a portion of the second picture relative to the first picture.

39. The device of claim 38, wherein the video decoder is configured to identify the first picture during decoding of a motion vector for a block of the second picture, and wherein the video decoder is configured to decode the motion vector according to at least one of advanced motion vector prediction (AMVP), temporal motion vector prediction (TMVP), and merge mode.

40. The device of claim 37, wherein the video decoder is further configured to decode, in accordance with an extension to the base video coding specification, a third picture based at least in part on the POC value and the second-dimension picture identifier of the first picture.

41. The device of claim 40, wherein the video decoder is configured to mark all inter-view reference pictures for the third picture, including the first picture, as long-term reference pictures prior to decoding the third picture, store a status for each of the inter-view reference pictures, wherein the status comprises one of long-term reference picture, short-term reference picture, and unused for reference prior to marking the inter-view reference pictures as long-term reference pictures, and, after decoding the third picture, set new statuses for each of the inter-view reference pictures based on the stored statuses.

42. The device of claim 40, wherein the video decoder is further configured to mark each inter-view reference picture for the third picture, including the first picture, as one of a long-term reference picture, a short-term reference picture, and unused for reference after decoding the third picture, assign each of the inter-view reference pictures a new POC value that is currently unused after marking an inter-view reference picture as a long-term reference picture, and restore an original POC value for the inter-view reference picture after decoding the second picture.

43. A device for encoding video data, the device comprising:

a memory configured to store video data; and

a video encoder configured to;

determine that a first motion vector of a first block of a picture refers to a short-term reference picture,

determine that a second motion vector of a second block of the picture refers to a long-term reference picture, and

based on the determination that the first motion vector refers to the short-term reference picture and the determination that the second motion vector refers to the long-term reference picture, disable motion vector prediction between the first motion vector of the first block of the picture and the second motion vector of the second block of the picture, wherein the first block and the second block are spatial neighbors in the picture, and wherein to disable motion vector prediction between the first motion vector and the second motion vector, the video encoder is configured to;

at least one of process a value for a flag indicating whether the second motion vector is available for use as a first motion vector predictor for the first motion value equal to a value indicating that the second motion vector is not available for use as the first motion vector predictor for the first motion vector or process a value for a flag indicating whether the first motion vector is available for use as a second motion vector predictor for the second motion value equal to a value indicating that the first motion vector is not available for use as the second motion vector predictor for the second motion vector; and

encode the first motion vector without using the second motion vector to predict the first motion vector and encode the second motion vector without using the first motion vector to predict the second motion vector.

44. The device of claim 43, wherein the video encoder is configured to enable prediction between a first short-term motion vector of the picture and a second short-term motion vector of the picture, and scale at least one of the first short-term motion vector and the second short-term motion vector based on a picture order count (POC) value for a first short-term reference picture referred to by the first short-term motion vector and a POC value for a second short-term reference picture referred to by the second short-term motion vector.

45. The device of claim 43, wherein the second-dimension picture identifier comprises at least one of a view identifier for a view including the first picture, a view order index for the view including the first picture, a combination of the view order index and a depth flag, a layer identifier for a scalable video coding (SVC) layer including the first picture, and a generic layer identifier.

46. The device of claim 43, wherein the video encoder is configured to encode a value indicating whether a second picture of the video data comprises a long-term reference picture, wherein the value indicating whether the second picture comprises the long-term reference picture further indicates whether the second picture is used for inter-view prediction.

47. The device of claim 43,

wherein to determine that the first motion vector of the first block of the second picture refers to the short-term reference picture, the video encoder is configured to determine that the first motion vector refers to a first reference picture marked as used for short-term reference, and

wherein to determine that the second motion vector of the second block of the second picture refers to the long-term reference picture, the video encoder is configured to determine that the second motion vector refers to a second reference picture marked as used for long-term reference.

48. The device of claim 43, wherein the picture comprises a second picture, and wherein the video encoder is further configured to:

encode data of the second picture that refers to a picture order count (POC) value for a first picture of video data,

encode data of the second picture that refers to a second-dimension picture identifier for the first picture, and

encode, in accordance with a base video coding specification, the second picture based at least in part on the POC value and the second-dimension picture identifier of the first picture.

49. The device of claim 48, wherein the video encoder is configured to identify the first picture using the POC value and the second-dimension picture identifier, and encode at least a portion of the second picture relative to the first picture.

50. The device of claim 49, wherein the video encoder is configured to identify the first picture during encoding of a motion vector for a block of the second picture, and wherein the video encoder is configured to encode the motion vector according to at least one of advanced motion vector prediction (AMVP), temporal motion vector prediction (TMVP), and merge mode.

51. The device of claim 48, wherein the video encoder is further configured to encode, in accordance with an extension to the base video coding specification, a third picture based at least in part on the POC value and the second-dimension picture identifier of the first picture.

52. The device of claim 51, wherein the video encoder is configured to mark all inter-view reference pictures for the third picture, including the first picture, as long-term reference pictures prior to encoding the third picture, store a status for each of the inter-view reference pictures, wherein the status comprises one of long-term reference picture, short-term reference picture, and unused for reference prior to marking the inter-view reference pictures as long-term reference pictures, and, after encoding the third picture, set new statuses for each of the inter-view reference pictures based on the stored statuses.

53. The device of claim 51, wherein the video encoder is further configured to mark each inter-view reference picture for the third picture, including the first picture, as one of a long-term reference picture, a short-term reference picture, and unused for reference after encoding the third picture, assign each of the inter-view reference pictures a new POC value that is currently unused after marking an inter-view reference picture as a long-term reference picture, and restore an original POC value for the inter-view reference picture after encoding the second picture.

54. A device for encoding video data, the device comprising:

means for determining that a first motion vector of a first block of a picture refers to a short-term reference picture;

means for determining that a second motion vector of a second block of the picture refers to a long-term reference picture; and

means for disabling, based on the determination that the first motion vector refers to the short-term reference picture and the determination that the second motion vector refers to the long-term reference picture, motion vector prediction between the first motion vector of the first block of the picture and the second motion vector of the second block of the picture, wherein the first block and the second block are spatial neighbors in the picture, and wherein the means for disabling motion vector prediction between the first motion vector and the second motion vector comprises;

at least one of means for processing a value for a flag indicating whether the second motion vector is available for use as a first motion vector predictor for the first motion value equal to a value indicating that the second motion vector is not available for use as the first motion vector predictor for the first motion vector or means for processing a value for a flag indicating whether the first motion vector is available for use as a second motion vector predictor for the second motion value equal to a value indicating that the first motion vector is not available for use as the second motion vector predictor for the second motion vector; and

means for encoding the first motion vector without using the second motion vector to predict the first motion vector and means for encoding the second motion vector without using the first motion vector to predict the second motion vector.

55. The device of claim 54,

wherein the means for determining that the first motion vector of the first block of the second picture refers to the short-term reference picture comprises means for determining that the first motion vector refers to a first reference picture marked as used for short-term reference, and

wherein the means for determining that the second motion vector of the second block of the second picture refers to the long-term reference picture comprises means for determining that the second motion vector refers to a second reference picture marked as used for long-term reference.

56. The device of claim 54, wherein the picture comprises a second picture, further comprising:

means for encoding data of the second picture that refers to a picture order count (POC) value for a first picture of video data;

means for encoding data of the second picture that refers to a second-dimension picture identifier for the first picture; and

means for encoding, in accordance with a base video coding specification, the second picture based at least in part on the POC value and the second-dimension picture identifier of the first picture.

57. A computer-readable storage medium having stored thereon instructions that, when executed, cause a processor to:

determine that a first motion vector of a first block of a picture refers to a short-term reference picture;

determine that a second motion vector of a second block of the picture refers to a long-term reference picture; and

based on the determination that the first motion vector refers to the short-term reference picture and the determination that the second motion vector refers to the long-term reference picture, disable motion vector prediction between the first motion vector of the first block of the picture and the second motion vector of the second block of the picture, wherein the first block and the second block are spatial neighbors in the picture, and wherein the instructions that cause the processor to disable motion vector prediction between the first motion vector and the second motion vector comprise instructions that cause the processor to;

at least one of process a value for a flag indicating whether the second motion vector is available for use as a first motion vector predictor for the first motion value equal to a value indicating that the second motion vector is not available for use as the first motion vector predictor for the first motion vector or process a value for a flag indicating whether the first motion vector is available for use as a second motion vector predictor for the second motion value equal to a value indicating that the first motion vector is not available for use as the second motion vector predictor for the second motion vector; and

decode the first motion vector without using the second motion vector to predict the first motion vector and decode the second motion vector without using the first motion vector to predict the second motion vector.

58. The computer-readable storage medium of claim 57,

wherein the instructions that cause the processor to determine that the first motion vector of the first block of the second picture refers to the short-term reference picture comprises instructions that cause the processor to determine that the first motion vector refers to a first reference picture marked as used for short-term reference, and

wherein the instructions that cause the processor to determine that the second motion vector of the second block of the second picture refers to the long-term reference picture comprises instructions that cause the processor to determine that the second motion vector refers to a second reference picture marked as used for long-term reference.

59. The computer-readable storage medium of claim 57, wherein the picture comprises a second picture, further comprising instructions that cause the processor to:

decode data of a second picture that refers to a picture order count (POC) value for a first picture of video data;

decode data of the second picture that refers to a second-dimension picture identifier for the first picture; and

decode, in accordance with a base video coding specification, the second picture based at least in part on the POC value and the second-dimension picture identifier of the first picture.