Generic use of HEVC SEI messages for multi-layer codecs

• US 9,894,370 B2
• Filed: 03/23/2015
• Issued: 02/13/2018
• Est. Priority Date: 03/24/2014
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Wang, Ye-Kui, Ramasubramonian, Adarsh Krishnan, Hendry, Fnu
• Primary Examiner(s): Beasley, Deirdre
• Application Number:
  US14/665,953
• Publication Number:
  US 20150271528A1
• Time in Patent Office: 1,058 Days
• Field of Search: 37524026
• CPC Class Codes: H04N 19/30 using hierarchical techniques, e.g. scalability H04N19/63 takes precedence
• CPC Class Codes: H04N 19/37 with arrangements for assigning different transmission priorities to video input data or to video coded data
• CPC Class Codes: H04N 19/60 using transform coding
• CPC Class Codes: H04N 19/70 characterised by syntax aspects related to video coding, e.g. related to compression standards
• CPC Class Codes: H04N 19/88 involving rearrangement of data among different coding units, e.g. shuffling, interleaving, scrambling or permutation of pixel data or permutation of transform coefficient data among different blocks
• CPC Class Codes: H04N 19/89 involving methods or arrangements for detection of transmission errors at the decoder

Abstract

In an example, a method of coding video data includes coding one or more non-video coding layer (VCL) network abstraction layer (NAL) units of a layer of a multi-layer bitstream of video data, where the one or more non-VCL NAL units contain an SEI message having an SEI payload type. The method also includes determining one or more syntax values of the multi-layer bitstream to which the SEI message applies based on the SEI payload type.

Claims

1. A method of coding video data, the method comprising:

coding one or more non-video coding layer (VCL) network abstraction layer (NAL) units of a multi-layer bitstream of video data containing one or more access units having a plurality of pictures that correspond to a single time instance, wherein the one or more non-VCL NAL units contain a scalable nesting supplemental enhancement information (SEI) message having an SEI payload type, wherein the scalable nesting SEI message contains one or more SEI messages; and

determining one or more syntax values of one or more syntax elements of the multi-layer bitstream to which the scalable nesting SEI message applies based on the SEI payload type,

wherein determining the one or more syntax values comprises determining, based on the SEI payload type being included in a first set of payload types, that a bitstream_subset_flag of the scalable nesting SEI message is equal to zero, wherein the bitstream_subset_flag equal to zero specifies that the one or more SEI messages contained in the scalable nesting SEI message apply to specific layers or sub-layers, and wherein the bitstream_subset_flag equal to one specifies that the one or more SEI messages contained in the scalable nesting SEI message apply to one or more sub-bitstreams resulting from a sub-bitstream extraction process.

2. The method of claim 1, wherein the first set of payload types includes payload types 2, 3, 6, 9, 15, 16, 17, 19, 22, 23, 45, 47, 128, 131, 132, and 134.

3. The method of claim 1, wherein determining the one or more syntax values further comprises determining, based on the SEI payload type being included in the first set of payload types, that the one or more non-VCL NAL units containing the scalable nesting SEI message has a TemporalId that is equal to zero and a maxTemporalId[i] that is equal to a maximum value of TemporalId for all values of i.

4. The method of claim 3, wherein the first set of payload types includes payload types 2, 3, 6, 9, 15, 16, 17, 19, 22, 23, 45, 47, 128, 131, 132, and 134.

5. The method of claim 1, further comprising coding an active parameter sets SEI message of the multi-layer bitstream only in a non-nested SEI message and not in the scalable nesting SEI message.

6. The method of claim 1, further comprising:

coding the bitstream_subset_flag syntax element of the multi-layer bitstream; and

based on the bitstream_subset_flag being equal to one and no layer sets specified by a video parameter set (VPS) of the multi-layer bitstream including layer identifiers in a range of zero to a layer identifier of the non-VCL NAL units containing the SEI message, inclusive, determining that a value of a default_op_flag syntax element of the multi-layer bitstream is zero valued.

7. The method of claim 1, further comprising:

coding a nesting_op_flag of the multi-layer bitstream and an all_layers_flag of the multi-layer bitstream; and

based on the nesting_op_flag having a value of zero and the all_layers_flag having a value of one, determining that a value of a maxTemporalId [0] syntax element of the multi-layer bitstream is equal to a maximum value of TemporalId.

8. The method of claim 1, further comprising:

coding the bitstream_subset_flag syntax element of the multi-layer bitstream and a nesting_op_flag syntax element of the multi-layer bitstream; and

based on the bitstream_subset_flag syntax element having a value of one and the nesting_op_flag syntax element having a value of zero, determining that a nestingLayeridList[0] of the multi-layer bitstream includes only layer identifier values of a layer set specified in a VPS of the multi-layer bitstream.

9. The method of claim 1, wherein coding comprises encoding, the method further comprising:

generating residual data for video data of a layer of the multi-layer bitstream indicating a difference between predictive data and the video data;

applying a transform to the residual data the generate transform coefficients; and

generating a bitstream that includes an indication of the transform coefficients.

10. The method of claim 1, wherein coding comprises decoding, the method further comprising:

obtaining transform coefficients for video data of a layer of the multi-layer bitstream from an encoded bitstream;

applying an inverse transform to the transform coefficients to generate residual data; and

determining the video data based on the generated residual data.

11. A device for coding video data, the device comprising:

a memory configured to store a layer of a multi-layer bitstream; and

one or more processors configured to;

code one or more non-video coding layer (VCL) network abstraction layer (NAL) units of a multi-layer bitstream of video data containing one or more access units having a plurality of pictures that correspond to a single time instance, wherein the one or more non-VCL NAL units contain a scalable nesting supplemental enhancement information (SEI) message having an SEI payload type, and wherein the scalable nesting SEI message contains one or more SEI messages; and

determine one or more syntax values of one or more syntax elements of the multi-layer bitstream to which the scalable nesting SEI message applies based on the SEI payload type,

wherein to determine the one or more syntax values, the one or more processors are configured to determine, based on the SEI payload type being included in a first set of payload types, that a bitstream_subset_flag of the scalable nesting SEI message is equal to zero, wherein the bitstream_subset_flag equal to zero specifies that the one or more SEI messages contained in the scalable nesting SEI message apply to specific layers or sub-layers, and wherein the bitstream_subset_flag equal to one specifies that the one or more SEI messages contained in the scalable nesting SEI message apply to one or more sub-bitstreams resulting from a sub-bitstream extraction process.

12. The device of claim 11, wherein the first set of payload types includes payload types 2, 3, 6, 9, 15, 16, 17, 19, 22, 23, 45, 47, 128, 131, 132, and 134.

13. The device of claim 11, wherein to determine the one or more syntax values, the one or more processors are configured to determine, based on the SEI payload type being included in the first set of payload types, that the one or more non-VCL NAL units containing the scalable nesting SEI message has a TemporalId that is equal to zero and a maxTemporalId[i] that is equal to a maximum value of TemporalId for all values of i.

14. The device of claim 13, wherein the first set of payload types includes payload types 2, 3, 6, 9, 15, 16, 17, 19, 22, 23, 45, 47, 128, 131, 132, and 134.

15. The device of claim 11, wherein the one or more processors are further configured to code an active parameter sets SEI message of the multi-layer bitstream only in a non-nested SEI message and not in the scalable nesting SEI message.

16. The device of claim 11, wherein the one or more processors are further configured to:

code the bitstream_subset_flag syntax element of the multi-layer bitstream; and

based on the bitstream_subset_flag being equal to one and no layer sets specified by a video parameter set (VPS) of the multi-layer bitstream including layer identifiers in a range of zero to a layer identifier of the non-VCL NAL units containing the SEI message, inclusive, determine that a value of a default_op_flag syntax element of the multi-layer bitstream is zero valued.

17. The device of claim 11, wherein the one or more processors are further configured to:

code a nesting_op_flag of the multi-layer bitstream and an all_layers_flag of the multi-layer bitstream; and

based on the nesting_op_flag having a value of zero and the all_layers_flag having a value of one, determine that a value of a maxTemporalId[0] syntax element of the multi-layer bitstream is equal to a maximum value of TemporalId.

18. The device of claim 11, wherein the one or more processors are further configured to:

code the bitstream_subset_flag syntax element of the multi-layer bitstream and a nesting_op_flag syntax element of the multi-layer bitstream; and

based on the bitstream_subset_flag syntax element having a value of one and the nesting_op_flag syntax element having a value of zero, determine that a nestingLayeridList[0] of the multi-layer bitstream includes only layer identifier values of a layer set specified in a VPS of the multi-layer bitstream.

19. The device of claim 11, wherein the device comprises at least one of:

an integrated circuit;

a microprocessor;

or

a wireless communication device.

20. An apparatus for coding video data, the apparatus comprising:

means for coding one or more non-video coding layer (VCL) network abstraction layer (NAL) units of a multi-layer bitstream of video data containing one or more access units having a plurality of pictures that correspond to a single time instance, wherein the one or more non-VCL NAL units contain a scalable nesting supplemental enhancement information (SEI) message having an SEI payload type, wherein the scalable nesting SEI message contains one or more SEI messages; and

means for determining one or more syntax values of one or more syntax elements of the multi-layer bitstream to which the scalable nesting SEI message applies based on the SEI payload type,

wherein the means for determining the one or more syntax values comprises means for determining, based on the SEI payload type being included in a first set of payload types, that a bitstream_subset_flag of the scalable nesting SEI message is equal to zero, wherein the bitstream_subset_flag equal to zero specifies that the one or more SEI messages contained in the scalable nesting SEI message apply to specific layers or sub-layers, and wherein the bitstream_subset_flag equal to one specifies that the one or more SEI messages contained in the scalable nesting SEI message apply to one or more sub-bitstreams resulting from a sub-bitstream extraction process.

21. A non-transitory computer-readable medium having instructions stored thereon that, when executed, cause one or more processors to:

code one or more non-video coding layer (VCL) network abstraction layer (NAL) units of a multi-layer bitstream of video data containing one or more access units having a plurality of pictures that correspond to a single time instance, wherein the one or more non-VCL NAL units contain a scalable nesting supplemental enhancement information (SEI) message having an SEI payload type, wherein the scalable nesting SEI message contains one or more SEI messages; and

determine one or more syntax values of one or more syntax elements of the multi-layer bitstream to which the scalable nesting SEI message applies based on the SEI payload type,

wherein to determine the one or more syntax values, the instructions further cause the one or more processors to determine, based on the SEI payload type being included in a first set of payload types, that a bitstream_subset_flag of the scalable nesting SEI message is equal to zero, wherein the bitstream_subset_flag equal to zero specifies that the one or more SEI messages contained in the scalable nesting SEI message apply to specific layers or sub-layers, and wherein the bitstream_subset_flag equal to one specifies that the one or more SEI messages contained in the scalable nesting SEI message apply to one or more sub-bitstreams resulting from a sub-bitstream extraction process.