Inter-layer dependency information for 3DV

• US 9,942,558 B2
• Filed: 04/30/2010
• Issued: 04/10/2018
• Est. Priority Date: 05/01/2009
• Status: Expired due to Fees

• Current Assignee: Interdigital Vc Holdings, Inc. (InterDigital, Inc.)
• Original Assignee: Thomson Licensing (Vantiva SA)
• Inventors: Tian, Dong, Lai, Wang Lin
• Primary Examiner(s): Holder, Anner N
• Application Number:
  US13/318,418
• Publication Number:
  US 20120056981A1
• Time in Patent Office: 2,902 Days
• Field of Search: 348 72
• CPC Class Codes: H04N 19/30 using hierarchical techniques, e.g. scalability H04N19/63 takes precedence
• CPC Class Codes: H04N 19/597 specially adapted for multi-view video sequence encoding
• CPC Class Codes: H04N 19/70 characterised by syntax aspects related to video coding, e.g. related to compression standards

Abstract

Various implementations are directed to providing inter-layer dependency information. In one implementation, syntax elements are generated that indicate an inter-layer dependency structure among three-dimensional video (3DV) layers. Based on the inter-layer dependency structure, an inter-layer reference is identified for a picture from a layer of the 3DV layers. The picture is encoded based, at least in part, on the inter-layer reference. Corresponding decoding implementations are also provided. Additionally, in another implementation, a transmission priority and an indication of network congestion are used to determine whether to transmit data for a particular 3DV layer. The transmission priority is based on an inter-layer dependency structure among multiple 3DV layers. Another implementation is directed to a network abstraction layer unit that can explicitly identify and convey inter-layer references and corresponding dependencies.

Claims

1. A method comprising:

generating syntax elements indicating an inter-layer dependency structure among three dimensional video (3DV) layers, the syntax elements relating to a syntax structure defining inter-layer dependencies among 3DV layers wherein the inter-layer dependency structure specifies at least one of;

b) that an inter-layer prediction reference for an occlusion video layer is the 2D video layer;

c) that an inter-layer prediction reference for an occlusion depth layer is the depth layer; and

d) that an inter-layer prediction reference for a transparency layer is the 2D video layer;

identifying, based on the inter-layer dependency structure, an inter-layer prediction reference for a picture from a layer of the 3DV layers; and

encoding the picture based, at least in part, on the inter-layer prediction reference.

2. The method of claim 1, further comprising:

generating a bitstream that includes the encoded picture; and

providing the encoded picture and the syntax elements for use in decoding the encoded picture.

3. The method of claim 2, wherein the syntax elements are separate from the bitstream.

4. The method of claim 2, wherein the syntax elements are included within the bitstream.

5. The method of claim 2, wherein the syntax elements are included within at least one of network abstraction layer (NAL) unit header, a sequence parameter set (SPS), a picture parameter set (PPS), supplemental enhancement information (SEI) or a slice header.

6. The method of claim 2, wherein the syntax elements explicitly convey the inter-layer dependency structure.

7. The method of claim 6, wherein the syntax elements provide 3DV layer identifiers for each given 3DV layer and wherein the 3DV layer identifiers indicate the inter-layer prediction references for a corresponding given 3DV layer.

8. The method of claim 1, wherein the inter-layer dependency structure is predefined and wherein the syntax elements indicate the inter-layer dependency structure by conveying whether particular 3DV layers are present for each 3DV view.

10. The method of claim 1, wherein the generating of the syntax elements, the identifying of the inter-layer prediction reference, and the encoding of the picture are performed at one or more encoders.

9. The method of claim wherein the inter-layer dependency structure specifies at least one of:

b) that inter-layer prediction references for an occlusion video layer are the 2D video layer and the depth layer;

c) that inter-layer prediction references for an occlusion depth layer are the 2D video layer and the depth layer; and

d) that inter-layer prediction references for a transparency layer are the 2D video layer and the depth layer.

11. A non-transitory processor readable medium having stored thereon instmctions for causing a processor to perform at least the following:

generating syntax elements indicating an inter-layer dependency structure among three dimensional video (3DV) layers, the syntax elements relating to a syntax structure defining inter-layer dependencies among 3DV layers wherein the inter-layer dependency structure specifies at least one of;

b) that an inter-layer prediction reference for anocclusionvideo layer is the 2D video layer;

c) that an inter-layer prediction reference for an occlusion depth layer is the depth layer; and

d) that an inter-layer prediction reference for a transparency layer is the 2D video layer;

identifying, based on the inter-layer dependency structure, an inter-layer prediction reference for a picture from a layer of the 3DV layers; and

encoding the picture based, at least in part, on the inter-layer prediction reference.

12. An apparatus, comprising a processor configured to perform at least the following:

generating syntax elements indicating an inter-layer dependency structure among three dimensional video (3DV) layers, the syntax elements relating to a syntax structure defining inter-layer dependencies among 3DV layers wherein the inter-layer dependency structure specifies at least one of;

b) that an inter-layer prediction reference thr an occlusion video layer is the 2D video layer;

c) that an inter-layer prediction reference for an occlusion depth layer is the depth layer; and

d) that an inter-layer prediction reference for a transparency layer is the 2D video layer;

identifying, based on the inter-layer dependency structure, an inter-layer prediction reference for a picture from a layer of the 3DV layers; and

encoding the picture based, at least in part, on the inter-layer prediction reference.

13. An apparatus comprising:

a three-dimensional video (3DV) reference buffer configured to generate syntax elements indicating an inter-layer dependency structure among 3DV layers, the syntax elements relating to a syntax structure defining inter-layer dependencies among 3DV layers, and configured to identify, based on the inter-layer dependency structure, an, inter-layer prediction reference for a picture from a layer of the 3DV layers wherein the inter-layer dependency structure specifies at least one of;

b) that an inter-layer prediction reference for an occlusion video layer is the 2D video layer;

c) that an inter-layer prediction reference for an occlusion depth layer is the depth layer; and

d) that an inter-layer prediction reference for a transparency layer is the 2D video laver; and

a 3DV layer encoder configured to encode the picture based, at least in part, on the inter-layer prediction reference.

14. The apparatus of claim 13 wherein the 3DV layer encoder is further configured to generate a bitstream that includes the encoded picture, and to provide the encoded picture and the syntax elements for use in decoding the encoded picture.

15. An apparatus comprising:

a three-dimensional video (3DV) reference buffer configured to generate syntax elements indicating an inter-layer dependency structure among 3DV layers, the syntax elements relating to a syntax structure defining inter-layer dependencies among 3DV layers, and configured, to identify, based on the inter-layer dependency structure, an inter-layer prediction reference for a picture from a layer of the 3DV layers wherein the inter-layer dependency structure specifies at least one of;

b) that an inter-layer prediction reference for an occlusion video layer is the 2D video layer;

c) that an it ier-layer prediction reference for an occlusion depth layer is the depth layer, and d) that an inter-layer prediction reference for a transparency layer is the 2D video layer;

a 3DV layer encoder configured to encode the picture based, at least in part, on the inter-layer prediction reference; and

a modulator configured to modulate a signal including the encoded picture.

16. A non-transitory processor readable medium having stored thereon a video signal structure, comprising:

one or more syntax portions for syntax elements indicating an inter-layer dependency structure among three-dimensional video (3DV) layers, the syntax elements relating to a syntax structure defining inter-layer dependencies among 3DV layers wherein the inter-layer dependency structure specifies at least one of;

b) that, an inter-layer prediction reference thr an occlusion video layer is the 2D video layer;

c) that an inter-layer prediction reference for an occlusion depth layer is the depth layer; and

d) that an inter-layer prediction reference for a transparency layer is the 2D video layer; and

one or more picture portions for an encoded picture that is encoded based, at least in part, on based on an inter-layer prediction reference from the inter-layer dependency structure, wherein the encoded picture and the syntax elements are provided for use in decoding the encoded picture.

17. A method comprising:

accessing an encoded picture from a bitstream, the picture describing three dimensional video (3DV) information for a particular 3DV layer, from a given view, at a given time;

accessing syntax elements indicating an inter-layer dependency structure for a set of 3DV layers that includes the particular 3DV layer, the syntax elements relating to a syntax structure defining inter-layer dependencies among 3DV layers wherein the inter-layer dependency structure specifies at least one of;

b) that an inter-layer prediction reference for an occlusion video layer is the 2D video layer;

c) that an inter-layer prediction reference for an occlusion depth layer is the depth layer; and

d) that an inter-layer prediction reference for a transparency layer is the 2D video layer; and

decoding the encoded picture based, at least in part, on the inter-layer dependency structure.

18. The method of claim 17, further comprising providing the decoded pictures in an output format that indicates the inter-layer dependency structure.

19. The method of claim 18, wherein the output format includes the syntax elements.

20. The method of claim 17, wherein the decoding further comprises constructing a reference picture list for the picture based on the syntax elements.

21. The method of claim 17, wherein the accessing of the encoded picture, the accessing of the syntax elements, and the decoding of the encoded picture are performed at one or more of an encoder or a decoder.

22. A non-transitory processor readable medium having stored thereon instructions for causing a processor to perform at least the following:

accessing an encoded picture from a bitstream, the picture describing three dimensional video (3DV) information for a particular 3DV layer, from a given view, at a given time;

accessing syntax elements indicating an inter-layer dependency structure for a set of 3DV layers that includes the particular 3DV layer, the syntax elements relating to a syntax structure defining inter-layer dependencies among 3DV layers wherein the inter-layer dependency structure specifies at least one of;

b) that an inter-layer prediction reference for an occlusion video layer is the 2D video layer;

c) that an inter-layer prediction reference for an occlusion depth layer is the depth layer, and

d) that an inter-layer prediction reference for a transparency layer is the 2D video layer; and

decoding the encoded picture based, at least in part, on the inter-layer dependency structure.

23. An apparatus, comprising a processor configured to perform at least the following:

accessing an encoded picture from a bitstream, the picture describing three dimensional video (3DV) information for a particular 3DV layer, from a given view, at a given time;

accessing syntax elements indicating an inter-layer dependency structure for a set of 3DV layers that includes the paniculat 3DV layer, the syntax elements relating to a syntax structure defining inter-layer dependencies among 3DV layers wherein the inter-layer dependency structure specifies at least one of;

b) that an inter-layer prediction reference for an occlusion video layer is the 2D video layer;

c) that an inter-layer prediction reference for an occlusion depth layer is the depth layer; and

d) that an inter-layer prediction reference for a transparency layer is the 2D video layer; and

decoding the encoded picture based, at least in part, on the inter-layer dependency structure.

24. An apparatus comprising:

a three-dimensional video (3DV) layer decoder configured to access an encoded picture from a bitstream, the picture describing three-dimensional video (3DV) information for a particular 3DV layer, from a given view, at a given time; and

a 3DV reference buffer configured to access syntax elements indicating an inter-layer dependency structure for a set of 3DV layers that includes the particular 3DV layer, the syntax elements relating to a syntax structure defining inter-layer dependencies among 3DV layers, wherein the 3DV layer decoder is further configured to decode the encoded picture based, at least in part, on the inter-layer dependency structure wherein the inter-layer dependency structure specifies at least one of;

b) that an inter-layer prediction reference torr an occlusion video layer is the 2D video layer;

c) that an inter-layer prediction reference for an occlusion depth layer is the depth layer; and

d) that an inter-layer prediction reference for a transparency layer is the 2D video layer.

25. An, apparatus comprising:

a demodulator configured to demodulate a signal to produce a demodulated bitstream;

a three-dimensional video (3DV) layer decoder configured to access an encoded picture from the demodulated bitstream, the picture describing three dimensional video (3DV) information for a particular 3DV layer, from a given view, at a given time; and

a 3DV reference buffer configured to access syntax elements indicating an inter-layer dependency structure for a set of 3DV layers that includes the particular 3DV layer, the syntax elements relating to a syntax structure defining inter-layer dependencies among 3DV layers, wherein the 3DV layer decoder is further configured to decode the, encoded picture based, at least in part, on the inter-layer dependency structure wherein the inter-layer dependency structure specifies at least one of;

b) that an inter-layer prediction reference for an occlusion video layer is the 2D video layer;

c) that an inter-layer prediction reference for an occlusion depth layer is the depth layer, and d) that an inter-layer prediction reference for a transparency layer is the 2D video layer.

26. A method comprising:

accessing a transmission priority for a particular three-dimensional video (3DV) layer of multiple 3DV layers, the transmission priority being based on an inter-layer dependency structure among the multiple 3DV layers wherein the inter-layer dependency structure specifies at least one of;

b) that an inter-layer prediction reference torr ark occlusion video layer is the 2D video layer;

c) that an utter-layer prediction reference for an occlusion depth layer is the depth layer; and

d) that an inter-layer prediction reference for a transparency layer is the 2D video layer, and

determining whether to transmit encoded data belonging to the particular 3DV layer, the determining of whether to transmit being based on the transmission priority tbr the particular 3DV layer and based on an indication of network congestion.

27. The method of claim 26, further comprising:

accessing syntax elements indicating the inter-layer dependency structure among the multiple 3DV layers, the syntax elements relating to a syntax structure defining inter-layer dependencies among 3DV layers; and

determining the transmission priority for the particular 3DV layer of the 3DV layers based on the structure.