Generic use of HEVC SEI messages for multi-layer codecs

• US 10,645,404 B2
• Filed: 03/23/2015
• Issued: 05/05/2020
• Est. Priority Date: 03/24/2014
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Wang, Ye-Kui, Ramasubramonian, Adarsh Krishnan, Hendry, Fnu
• Primary Examiner(s): Atala, Jamie J
• Assistant Examiner(s): Demosky, Patrick E
• Application Number:
  US14/665,999
• Publication Number:
  US 20150271529A1
• Time in Patent Office: 1,870 Days
• Field of Search: 37524026
• CPC Class Codes: H04N 19/30 using hierarchical techniques, e.g. scalability H04N19/63 takes precedence
• CPC Class Codes: H04N 19/37 with arrangements for assigning different transmission priorities to video input data or to video coded data
• CPC Class Codes: H04N 19/60 using transform coding
• CPC Class Codes: H04N 19/70 characterised by syntax aspects related to video coding, e.g. related to compression standards
• CPC Class Codes: H04N 19/88 involving rearrangement of data among different coding units, e.g. shuffling, interleaving, scrambling or permutation of pixel data or permutation of transform coefficient data among different blocks
• CPC Class Codes: H04N 19/89 involving methods or arrangements for detection of transmission errors at the decoder

Abstract

In an example, a method of coding video data includes obtaining one or more video coding layer (VCL) network abstraction layer (NAL) units of an access unit and a first layer of a multi-layer bitstream of video data. The method also includes only coding one or more non-VCL NAL units containing an SEI message applicable to the VCL NAL units of the first layer together with the VCL NAL units of the first layer such that within the access unit the bitstream does not contain any coded pictures of any other layer of the multi-layer bitstream between the VCL NAL units of the first layer and the non-VCL NAL units containing the SEI message applicable to the VCL NAL units of the first layer.

Claims

1. A method of coding video data, the method comprising:

obtaining an access unit of a multi-layer bitstream of the video data that includes a first layer and at least one second layer, wherein obtaining the access unit comprises obtaining one or more video coding layer (VCL) network abstraction layer (NAL) units of the first layer and obtaining one or more VCL NAL units of the at least one second layer; and

coding the access unit, wherein coding the access unit comprises coding one or more first non-VCL NAL units containing a first SEI message applicable to the one or more VCL NAL units of the first layer together with the one or more VCL NAL units of the first layer, and coding one or more second non-VCL NAL units containing a second SEI message applicable to the one or more VCL NAL units of the at least one second layer, such that the bitstream does not contain any coded pictures of any layer other than the first layer of the multi-layer bitstream between the one or more VCL NAL units of the first layer and the one or more first non-VCL NAL units, and does not contain any coded pictures of any layer other than the at least one second layer between the one or more VCL NAL units of the at least one second layer and the one or more second non-VCL NAL units,

wherein the one or more VCL NAL units of the first layer are VCL NAL units of a first picture of the first layer and the first SEI message comprises a first prefix SEI message applicable to the first picture of the first layer, and wherein coding the one or more first non-VCL NAL units containing the first prefix SEI message together with the one or more VCL NAL units of the first layer comprises coding the one or more first non-VCL NAL units preceding the one or more VCL NAL units of the first layer in the bitstream, and

wherein the one or more VCL NAL units of the at least one second layer are VCL NAL units of a second picture of the at least one second layer and the second SEI message comprises a second prefix SEI message applicable to the second picture of the at least one second layer, the method further comprising coding the one or more second non-VCL NAL units containing the second prefix SEI message following the first picture in the bitstream.

2. The method of claim 1, wherein coding the one or more first non-VCL NAL units comprises coding the one or more first non-VCL NAL units directly adjacent to the one or more VCL NAL units of the first layer in the multi-layer bitstream, the method further comprising coding the one or more second non-VCL NAL units directly adjacent to the one or more VCL NAL units of the at least one second layer in the multi-layer bitstream.

3. The method of claim 1, wherein the one or more VCL NAL units of the first layer are VCL NAL units of a first picture of the first layer and the first SEI message comprises a first suffix SEI message applicable to the first picture of the first layer, and wherein coding the one or more first non-VCL NAL units containing the first suffix SEI message together with the one or more VCL NAL units of the first layer comprises coding the one or more first non-VCL NAL units subsequent to the one or more VCL NAL units of the first layer in the bitstream.

4. The method of claim 3, wherein the one or more VCL NAL units of the second layer are VCL NAL units of a second picture of the at least one second layer and the second SEI message comprises a second suffix SEI message applicable to the VCL NAL units of the second picture, the method further comprising:

coding the one or more second non-VCL NAL units following the second picture in the bitstream.

5. The method of claim 1, further comprising determining a maximum repetition parameter for the first SEI message based on a picture unit that contains VCL NAL units of a first picture of the first layer and associated non-VCL NAL units of the first picture, wherein the maximum repetition parameter for the first SEI message represents a maximum number of times that the respective SEI message may be repeated.

6. The method of claim 1, further comprising:

generating residual data for the one or more VCL NAL units of the first layer indicating a difference between predictive data and actual video data;

applying a transform to the residual data the generate transform coefficients; and

generating the multi-layer bitstream that includes an indication of the transform coefficients.

7. The method of claim 1, further comprising:

obtaining transform coefficients for the one or more VCL NAL units of the first layer from the multi-layer bitstream;

applying an inverse transform to the transform coefficients to generate residual data; and

determining the video data based on the generated residual data.

20. The method of claim 1, the method being executable on a wireless communication device, wherein the device comprises:

a memory configured to store the access unit of video data;

a processor configured to process the access unit of video data stored in said memory; and

a receiver configured to receive the access unit of video data.

21. The method of claim 20, wherein the wireless communication device is a cellular telephone and the access unit is received by the receiver and modulated according to a cellular communication standard.

8. A device for coding video data, the device comprising:

a memory configured to store at least a portion of a multi-layer bitstream of video data; and

one or more processors configured to;

obtain an access unit of a multi-layer bitstream of the video data that includes a first layer and at least one second layer, wherein to obtain the access unit, the one or more processors are configured to obtain one or more video coding layer (VCL) network abstraction layer (NAL) units of the first layer and obtain one or more VCL NAL units of the at least one second layer; and

code the access unit, wherein to code the access unit the one or more processors are configured to code one or more first non-VCL NAL units containing a first SEI message applicable to the one or more VCL NAL units of the first layer together with the one or more VCL NAL units of the first layer, and code one or more second non-VCL NAL units containing a second SEI message applicable to the one or more VCL NAL units of the at least one second layer, such that the bitstream does not contain any coded pictures of any layer other than the first layer of the multi-layer bitstream between the one or more VCL NAL units of the first layer and the one or more first non-VCL NAL units, and does not contain any coded pictures of any layer other than the at least one second layer between the one or more VCL NAL units of the at least one second layer and the one or more second non-VCL NAL units,

wherein the one or more VCL NAL units of the first layer are VCL NAL units of a first picture of the first layer and the first SEI message comprises a first prefix SEI message applicable to the first picture of the first layer, and wherein to code the one or more first non-VCL NAL units containing the first prefix SEI message together with the one or more VCL NAL units of the first layer, the one or more processors are configured to code the one or more first non-VCL NAL units preceding the one or more VCL NAL units of the first layer in the bitstream, and

wherein the one or more VCL NAL units of the at least one second layer are VCL NAL units of a second picture of the at least one second layer and the second SEI message comprises a second prefix SEI message applicable to the second picture of the at least one second layer, and wherein the one or more processors are further configured to code the one or more second non-VCL NAL units containing the second prefix SEI message following the first picture in the bitstream.

9. The device of claim 8, wherein to code the one or more first non-VCL NAL units, the one or more processors are configured to code the one or more first non-VCL NAL units directly adjacent to the one or more VCL NAL units of the first layer in the multi-layer bitstream, and wherein the one or more processors are further configured to code the one or more second non-VCL NAL units directly adjacent to the one or more VCL NAL units of the at least one second layer in the multi-layer bitstream.

10. The device of claim 8, wherein the one or more VCL NAL units of the first layer are VCL NAL units of a first picture of the first layer and the first SEI message comprises a first suffix SEI message applicable to the first picture of the first layer, and wherein to code the one or more first non-VCL NAL units containing the first suffix SEI message together with the one or more VCL NAL units of the first layer, the one or more processors are configured to code the one or more first non-VCL NAL units subsequent to the one or more VCL NAL units of the first layer in the bitstream.

11. The device of claim 10, wherein the one or more VCL NAL units of the second layer are VCL NAL units of a second picture of the at least one second layer and the second SEI message comprises a second suffix SEI message applicable to the VCL NAL units of the second picture, and wherein the one or more processors are further configured to:

code the one or more second non-VCL NAL units following the second picture in the bitstream.

12. The device of claim 8, wherein the one or more processors are further configured to determine a maximum repetition parameter for the first SEI message based on a picture unit that contains VCL NAL units of a first picture of the first layer and associated non-VCL NAL units of the first picture, and wherein the maximum repetition parameter for the first SEI message represents a maximum number of times that the respective SEI message may be repeated.

13. The device of claim 8, wherein the one or more processors are further configured to:

generate residual data for the one or more VCL NAL units of the first layer indicating a difference between predictive data and actual video data;

apply a transform to the residual data the generate transform coefficients; and

generate the multi-layer bitstream that includes an indication of the transform coefficients.

14. The device of claim 8, wherein the one or more processors are further configured to:

obtain transform coefficients for the one or more VCL NAL units of the first layer from the multi-layer bitstream;

apply an inverse transform to the transform coefficients to generate residual data; and

determine the video data based on the generated residual data.

15. The device of claim 8, wherein the device comprises at least one of:

an integrated circuit;

a microprocessor;

or

a wireless communication device.

22. The device of claim 8, wherein the device is a wireless communication device, further comprising a receiver configured to receive the access unit of video data.

23. The device of claim 22, wherein the wireless communication device is a cellular telephone and the access unit of video data is received by the receiver and modulated according to a cellular communication standard.

16. An apparatus for coding video data, the apparatus comprising:

means for obtaining an access unit of a multi-layer bitstream of the video data that includes a first layer and at least one second layer, wherein the means for obtaining the access unit comprises means for obtaining one or more video coding layer (VCL) network abstraction layer (NAL) units of the first layer and for obtaining one or more VCL NAL units of the at least one second layer; and

means for coding the access unit, wherein the means for coding the access unit comprises means for coding one or more first non-VCL NAL units containing a first SEI message applicable to the one or more VCL NAL units of the first layer together with the one or more VCL NAL units of the first layer, and coding one or more second non-VCL NAL units containing a second SEI message applicable to the one or more VCL NAL units of the at least one second layer, such that the bitstream does not contain any coded pictures of any layer other than the first layer of the multi-layer bitstream between the one or more VCL NAL units of the first layer and the one or more first non-VCL NAL units, and does not contain any coded pictures of any layer other than the at least one second layer between the one or more VCL NAL units of the at least one second layer and the one or more second non-VCL NAL units,

wherein the one or more VCL NAL units of the first layer are VCL NAL units of a first picture of the first layer and the first SEI message comprises a first prefix SEI message applicable to the first picture of the first layer, and wherein the means for coding the one or more first non-VCL NAL units containing the first prefix SEI message together with the one or more VCL NAL units of the first layer comprises means for coding the one or more first non-VCL NAL units preceding the one or more VCL NAL units of the first layer in the bitstream,

and wherein the one or more VCL NAL units of the at least one second layer are VCL NAL units of a second picture of the at least one second layer and the second SEI message comprises a second prefix SEI message applicable to the second picture of the at least one second layer, the apparatus further comprising means for coding the one or more second non-VCL NAL units containing the second prefix SEI message following the first picture in the bitstream.

17. The apparatus of claim 16, wherein the means for coding the one or more first non-VCL NAL units comprises means for coding the one or more first non-VCL NAL units directly adjacent to the one or more VCL NAL units of the first layer in the multi-layer bitstream, and further comprising means for coding the one or more second non-VCL NAL units directly adjacent to the one or more VCL NAL units of the at least one second layer in the multi-layer bitstream.

18. The apparatus of claim 16, wherein the one or more VCL NAL units of the first layer are VCL NAL units of a first picture of the first layer and the first SEI message comprises a first suffix SEI message applicable to the first picture of the first layer, and wherein the means for coding the one or more first non-VCL NAL units containing the first suffix SEI message together with the one or more VCL NAL units of the first layer comprises means for coding the one or more first non-VCL NAL units subsequent to the one or more VCL NAL units of the first layer in the bitstream.

19. A non-transitory computer-readable medium having instructions stored thereon that, when executed, cause one or more processors to:

obtain an access unit of a multi-layer bitstream of the video data that includes a first layer and at least one second layer, wherein to obtain the access unit the instructions cause the one or more processors to obtain one or more video coding layer (VCL) network abstraction layer (NAL) units of the first layer and obtain one or more VCL NAL units of the at least one second layer; and

code the access unit, wherein to code the access unit, the instructions cause the one or more processors to code one or more first non-VCL NAL units containing a first SEI message applicable to the one or more VCL NAL units of the first layer together with the one or more VCL NAL units of the first layer, and code one or more second non-VCL NAL units containing a second SEI message applicable to the one or more VCL NAL units of the at least one second layer, such that the bitstream does not contain any coded pictures of any layer other than the first layer of the multi-layer bitstream between the one or more VCL NAL units of the first layer and the one or more first non-VCL NAL units, and does not contain any coded pictures of any layer other than the at least one second layer between the one or more VCL NAL units of the at least one second layer and the one or more second non-VCL NAL units,

wherein the one or more VCL NAL units of the first layer are VCL NAL units of a first picture of the first layer and the first SEI message comprises a first prefix SEI message applicable to the first picture of the first layer, and wherein to code the one or more first non-VCL NAL units containing the first prefix SEI message together with the one or more VCL NAL units of the first layer, the instructions further cause the one or more processors to code the one or more first non-VCL NAL units preceding the one or more VCL NAL units of the first layer in the bitstream, and

wherein the one or more VCL NAL units of the at least one second layer are VCL NAL units of a second picture of the at least one second layer and the second SEI message comprises a second prefix SEI message applicable to the second picture of the at least one second layer, and wherein the instructions further cause the one or more processors to code the one or more second non-VCL NAL units containing the second prefix SEI message following the first picture in the bitstream.