3D VIDEO CODING FORMATS

• US 20120044322A1
• Filed: 04/30/2010
• Published: 02/23/2012
• Est. Priority Date: 05/01/2009
• Status: Abandoned Application

• Current Assignee: Thomson Licensing (Vantiva SA)
• Original Assignee: Thomson Licensing (Vantiva SA)
• Inventors: Tian, Dong, Lai, Wang Lin
• Application Number:
  US13/138,956
• Publication Number:
  US 20120044322A1
• Time in Patent Office: Days
• US Class Current: 348/43
• CPC Class Codes: H04N 19/30 using hierarchical techniques, e.g. scalability H04N19/63 takes precedence
• CPC Class Codes: H04N 19/597 specially adapted for multi-view video sequence encoding
• CPC Class Codes: H04N 19/70 characterised by syntax aspects related to video coding, e.g. related to compression standards

Abstract

Several implementations relate to 3D video (3DV) coding formats. One implementation encodes multiple pictures that describe different three-dimensional (3D) information for a given view at a given time. Syntax elements are generated that indicate, for the encoded multiple pictures, how the encoded picture fits into a structure that supports 3D processing. The structure defines content types for the multiple pictures. A bitstream is generated that includes the encoded multiple pictures and the syntax elements. The inclusion of the syntax elements provides, at a coded-bitstream level, indications of relationships between the encoded multiple pictures in the structure. The syntax elements also enable efficient inter-layer coding of the 3DV content, thereby reducing the bandwidth used to transmit the 3DV content. Corresponding decoding implementations are also provided. Extraction methods are also provided from extracting pictures of interest from such a 3DV multiple pictures and the syntax elements, the video stream characterized by such a 3D structure.

Claims

1. A method comprising:

encoding multiple pictures, the multiple pictures describing different three-dimensional (3D) information for a given view at a given time;

generating syntax elements that indicate, for the encoded multiple pictures, how the encoded picture fits into a structure that supports 3D processing, the structure defining content types for the multiple pictures; and

generating a bitstream that includes the encoded multiple pictures and the syntax elements, the inclusion of the syntax elements providing at a coded-bitstream level indications of relationships between the encoded multiple pictures in the structure.

2. The method of claim 1, further comprising:

encoding multiple additional pictures, the multiple additional pictures describing different 3D information for a second view at the given time, the second view being different from the given view; and

generating additional syntax elements that indicate, for the encoded multiple additional pictures, how the encoded picture fits into the structure that supports 3D processing, the structure including content types for the multiple additional pictures, and

wherein the generated bitstream further includes the encoded multiple additional pictures and the additional syntax elements, the inclusion of the additional syntax elements providing at a coded-bitstream level indications of relationships between the encoded additional multiple pictures in the structure.

3. The method of claim 1, wherein the syntax elements and structure are compatible with Multiview Video Coding (MVC).

4. The method of claim 3, wherein the syntax elements include an MVC view identifier for each of the multiple pictures, and dependencies between the multiple pictures are indicated by MVC sequence parameter set data.

5. The method of claim 1, wherein the syntax elements include a Network Abstraction Layer (NAL) unit type that indicates that the structure was used in the bitstream.

6. The method of claim 1, wherein the content types include at least two-dimensional video and depth.

7. The method of claim 1, wherein the content types include one or more of two-dimensional video, depth, occlusion video, occlusion depth, or transparency.

8. The method of claim 1, wherein the syntax elements for each of the encoded pictures comprise a 3D view identifier and a content type identifier, wherein the 3D view identifier identifies a 3D view composed of a set of the encoded pictures, each of the encoded pictures of the set corresponding to a different content type.

9. The method of claim 8, wherein the syntax elements for each of the encoded pictures are transmitted in a Network Abstraction Layer (NAL) unit separate from one or more NAL units including data for a corresponding encoded picture.

10. The method of claim 8, wherein the syntax elements for each of the encoded pictures comprise a different Multiview Video Coding (MVC) view identifier, thereby defining each encoded picture as a separate MVC view.

11. The method of claim 1, wherein the syntax elements for each of the encoded pictures comprises a Multiview Video Coding (MVC) view identifier which indicates both a 3D view and a content type, wherein the MVC view identifier identifies a 3D view composed of a set of the encoded pictures, each of the encoded pictures of the set corresponding to a different content type.

12. The method of claim 1, wherein:

the multiple pictures include a two-dimensional (2D) video picture, and one or more 3D video supplemental layer pictures,

the syntax elements for the 2D video encoded picture comprise a Multiview Video Coding (MVC) view identifier for the 2D video encoded picture, and

the syntax elements for the 2D video encoded picture and for the one or more 3D video supplemental layer encoded pictures comprise a 3D view identifier and a content type identifier for the 2D video encoded picture and for the corresponding 3D video supplemental layer encoded picture, wherein the 3D view identifier identifies a 3D view composed of a set of the encoded pictures, each of the encoded pictures of the set corresponding to a different content type.

13. The method of claim 12, wherein the syntax elements for the one or more 3D video supplemental layer pictures are transmitted in a Network Abstraction Layer (NAL) unit that includes data for a corresponding encoded picture.

14. The method of claim 12, wherein the syntax elements for the 2D video encoded picture is compatible with MVC and the syntax elements for the 3D video supplemental layers are compatible with a coding standard that is different from MVC.

15. The method of claim 1, wherein encoding the multiple pictures comprises encoding a picture of the multiple pictures using a different picture of the multiple pictures as a reference, thereby providing inter-layer coding between pictures of different content types.

16. The method of claim 1, wherein the syntax elements include one or more of (1) a Multiview Video Coding (MVC) view identifier, (2) a 3D video (3DV) view identifier, or (3) a 3DV layer identifier.

17. The method of claim 1, wherein the encoding, the generating of syntax elements, and the generating of the bitstream are performed at an encoder.

18. An apparatus comprising:

means for encoding multiple pictures, the multiple pictures describing different three-dimensional (3D) information for a given view at a given time;

means for generating syntax elements that indicate, for the encoded multiple pictures, how the encoded picture fits into a structure that supports 3D processing, the structure defining content types for the multiple pictures; and

means for generating a bitstream that includes the encoded multiple pictures and the syntax elements, the inclusion of the syntax elements providing at a coded-bitstream level indications of relationships between the encoded multiple pictures in the structure.

19. A processor readable medium having stored thereon instructions for causing a processor to perform at least the following:

encoding multiple pictures, the multiple pictures describing different three-dimensional (3D) information for a given view at a given time;

generating syntax elements that indicate, for the encoded multiple pictures, how the encoded picture fits into a structure that supports 3D processing, the structure defining content types for the multiple pictures; and

generating a bitstream that includes the encoded multiple pictures and the syntax elements, the inclusion of the syntax elements providing at a coded-bitstream level indications of relationships between the encoded multiple pictures in the structure.

20. An apparatus, comprising a processor configured to perform at least the following:

encoding multiple pictures, the multiple pictures describing different three-dimensional (3D) information for a given view at a given time;

generating syntax elements that indicate, for the encoded multiple pictures, how the encoded picture fits into a structure that supports 3D processing, the structure defining content types for the multiple pictures; and

generating a bitstream that includes the encoded multiple pictures and the syntax elements, the inclusion of the syntax elements providing at a coded-bitstream level indications of relationships between the encoded multiple pictures in the structure.

21. An apparatus comprising:

one or more encoders configured to encode multiple pictures, the multiple pictures describing different three-dimensional (3D) information for a given view at a given time; and

a 3D video reference buffer configured to generate syntax elements that indicate, for the encoded multiple pictures, how the encoded picture fits into a structure that supports 3D processing, the structure defining content types for the multiple pictures,

wherein the one or more encoders are further configured to generate a bitstream that includes the encoded multiple pictures and the syntax elements, the inclusion of the syntax elements providing at a coded-bitstream level indications of relationships between the encoded multiple pictures in the structure.

22. An apparatus comprising:

one or more encoders configured to encode multiple pictures, the multiple pictures describing different three-dimensional (3D) information for a given view at a given time;

a 3D video reference buffer configured to generate syntax elements that indicate, for the encoded multiple pictures, how the encoded picture fits into a structure that supports 3D processing, the structure defining content types for the multiple pictures,

wherein the one or more encoders are further configured to generate a bitstream that includes the encoded multiple pictures and the syntax elements, the inclusion of the syntax elements providing at a coded-bitstream level indications of relationships between the encoded multiple pictures in the structure; and

a modulator configured to modulate a signal including the generated bitstream.

23. (canceled)

24. (canceled)

25. A processor readable medium having stored thereon a video signal structure, comprising:

one or more picture portions for multiple encoded pictures, the multiple encoded pictures describing different three-dimensional (3D) information for a given view at a given time; and

one or more syntax portions for syntax elements that indicate, for the encoded multiple pictures, how the encoded picture fits into a 3D structure that supports 3D processing, the 3D structure defining content types for the multiple pictures, wherein the inclusion of the syntax elements provides at a coded-bitstream level indications of relationships between the encoded multiple pictures in the 3D structure.

26. A method comprising:

accessing encoded multiple pictures from a bitstream, the multiple pictures describing different three-dimensional (3D) information for a given view at a given time;

accessing syntax elements from the bitstream, the syntax elements indicating for the encoded multiple pictures how the encoded picture fits into a structure that supports 3D processing, the structure providing a defined relationship between the multiple pictures;

decoding the encoded multiple pictures; and

providing the decoded pictures in an output format that indicates the defined relationship between the multiple pictures.

27. The method of claim 26, wherein the output format includes syntax elements that specify how a given decoded picture fits into the structure.

28. The method of claim 26, wherein the decoding further comprises

using the defined relationship to render an additional picture that references one or more of a two-dimensional (2D) layer picture, a depth layer picture, an occlusion video layer picture, and occlusion depth layer picture, or a transparency picture.

29. The method of claim 26, wherein the accessing of the encoded multiple pictures, the accessing of the syntax elements, the decoding, and the providing are performed at one or more of an encoder or decoder.

30. The method of claim 26, further comprising:

identifying a two-dimensional (2D) video picture from the multiple pictures using the syntax elements;

identifying one or more of a depth picture, an occlusion video picture, an occlusion depth picture, or a transparency picture, from the multiple pictures using the syntax elements; and

performing 3D processing by rendering a new picture for an additional view based on the 2D video picture and the one or more pictures identified from the depth picture, the occlusion video picture, the occlusion depth picture, or the transparency picture.

31. An apparatus comprising:

means for accessing encoded multiple pictures from a bitstream, the multiple pictures describing different three-dimensional (3D) information for a given view at a given time;

means for accessing syntax elements from the bitstream, the syntax elements indicating for the encoded multiple pictures how the encoded picture fits into a structure that supports 3D processing, the structure providing a defined relationship between the multiple pictures;

means for decoding the encoded multiple pictures; and

means for providing the decoded pictures in an output format that indicates the defined relationship between the multiple pictures.

32. A processor readable medium having stored thereon instructions for causing a processor to perform at least the following:

accessing encoded multiple pictures from a bitstream, the multiple pictures describing different three-dimensional (3D) information for a given view at a given time;

accessing syntax elements from the bitstream, the syntax elements indicating for the encoded multiple pictures how the encoded picture fits into a structure that supports 3D processing, the structure providing a defined relationship between the multiple pictures;

decoding the encoded multiple pictures; and

providing the decoded pictures in an output format that indicates the defined relationship between the multiple pictures.

33. An apparatus, comprising a processor configured to perform at least the following:

accessing encoded multiple pictures from a bitstream, the multiple pictures describing different three-dimensional (3D) information for a given view at a given time;

accessing syntax elements from the bitstream, the syntax elements indicating for the encoded multiple pictures how the encoded picture fits into a structure that supports 3D processing, the structure providing a defined relationship between the multiple pictures;

decoding the encoded multiple pictures; and

providing the decoded pictures in an output format that indicates the defined relationship between the multiple pictures.

34. An apparatus comprising:

one or more decoders configured to access encoded multiple pictures from a bitstream, the multiple pictures describing different three-dimensional (3D) information for a given view at a given time; and

a 3D video reference buffer configured to access syntax elements from the bitstream, the syntax elements indicating for the encoded multiple pictures how the encoded picture fits into a structure that supports 3D processing, the structure providing a defined relationship between the multiple pictures,

wherein the one or more decoders are configured to decode the encoded multiple pictures, and

wherein the 3D video reference buffer is configured to provide the decoded pictures in an output format that indicates the defined relationship between the multiple pictures.

35. An apparatus comprising:

a demodulator configured to demodulate a signal including a bitstream;

one or more decoders configured to access encoded multiple pictures from the demodulated bitstream, the multiple pictures describing different three-dimensional (3D) information for a given view at a given time; and

a 3D video reference buffer configured to access syntax elements from the demodulated bitstream, the syntax elements indicating for the encoded multiple pictures how the encoded picture fits into a structure that supports 3D processing, the structure providing a defined relationship between the multiple pictures,

wherein the one or more decoders are configured to decode the encoded multiple pictures, and

wherein the 3D video reference buffer is configured to provide the decoded pictures in an output format that indicates the defined relationship between the multiple pictures.

36. A method comprising:

accessing syntax elements from a set of data, the syntax elements indicating how encoded pictures fit into a structure that supports 3D processing, the structure defining content types for the encoded pictures; and

extracting particular ones of the encoded pictures from the set of data, the particular ones of the encoded pictures corresponding to pictures that are from one or more given views of interest and that have one or more given content types of interest, or corresponding to a reference for a picture from a given view and a given content type of interest,

wherein the extracting of the pictures is based on the syntax elements and the indicated structure.

37. The method of claim 36, wherein the reference for the picture from the given view and the given content type of interest is from the given view and has a content type different from the given content type.