Providing frame packing type information for video coding

• US 9,602,802 B2
• Filed: 09/06/2013
• Issued: 03/21/2017
• Est. Priority Date: 07/21/2010
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Chen, Ying, Karczewicz, Marta, Wang, Yong
• Primary Examiner(s): Ren, Zhubing
• Application Number:
  US14/019,944
• Publication Number:
  US 20140009578A1
• Time in Patent Office: 1,292 Days
• US Class Current: 1/1
• CPC Class Codes: H04N 13/161 Encoding, multiplexing or demultiplexing different image signal components for multi-view video sequence encoding H04N19/597
• CPC Class Codes: H04N 19/597 specially adapted for multi-view video sequence encoding
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 19/70 characterised by syntax aspects related to video coding, e.g. related to compression standards
• CPC Class Codes: H04N 21/816 involving special video data, e.g 3D video
• CPC Class Codes: H04N 21/84 Generation or processing of descriptive data, e.g. content descriptors systems specially adapted for using meta-information in broadcast systems H04H60/73
• CPC Class Codes: H04N 21/8456 by decomposing the content in the time domain, e.g. in time segments
• CPC Class Codes: H04N 21/85406 involving a specific file format, e.g. MP4 format

Abstract

This disclosure includes techniques for signaling characteristics of a representation of multimedia content at a representation layer, such as frame packing arrangement information for the representation. In one example, an apparatus for receiving video data includes a processing unit configured to receive information indicative of whether a bitstream includes a packed frame of video data, wherein the packed frame comprises two frames corresponding to different views of a scene for three-dimensional playback, and wherein the information is present in a representation layer external to a codec layer of the bitstream, automatically determine whether the apparatus is capable of decoding and rendering the bitstream based on an analysis of the received information and decoding and rendering capabilities of the device, and retrieve the bitstream when the processing unit determines that the device is capable of decoding and rendering the bitstream.

Claims

1. A method of providing video data, the method comprising:

receiving, by a server device, a packed frame of a representation of a plurality of representations of video data of multimedia content, wherein the packed frame comprises two frames corresponding to different views of a scene for three-dimensional playback, wherein the representations act as alternatives to each other and have different coding or rendering characteristics from each other, wherein each of the representations corresponds to a separate, respective bitstream, and wherein each of the representations includes a respective plurality of segments including coded video data, the segments comprising respective independently retrievable files;

providing, by the server device, a manifest file to a client device separate from the server device, wherein the manifest file indicates a frame packing arrangement for the packed frame, wherein the manifest file includes descriptive data for each of the representations, and wherein the manifest file is separate from the segments of the representations such that the manifest file does not include any coded video data;

receiving, by the server device, a request from the client device for at least a portion of one of the segments of one of the representations described by the manifest file; and

in response to the request, sending, by the server device, at least the portion of the one of the segments to the client device.

2. The method of claim 1, wherein providing the manifest file comprises providing the manifest file in response to a request from the client device for representation layer data.

3. The method of claim 1, wherein providing the manifest file comprises:

forming a media presentation description data structure that describes three-dimensional video characteristics of each of the plurality of representations; and

sending the media presentation description data structure to the client device.

4. The method of claim 1, further comprising:

receiving a request for data from a representation of multimedia content, the representation comprising the packed frame, from the client device when the client device is configured to decode packed frames having the frame packing arrangement of the packed frame; and

in response to the request, sending the data to the client device.

5. The method of claim 1, further comprising sending information at a codec layer for a representation of multimedia content, the representation comprising the packed frame, to the client device, wherein the information comprises a frame packing arrangement supplemental enhancement information (SEI) message, and wherein the frame packing arrangement SEI message is separate from the manifest file.

6. The method of claim 1, further comprising providing an indication of an aspect sub-sampling ratio for each of the views.

7. The method of claim 6, wherein providing the indication of the aspect sub-sampling ratio comprises including information for the indication in at least one of a codec layer, a file format layer in a media file container format, and a transport layer.

8. The method of claim 7, wherein the transport layer comprises one of MPEG-2 Transport Stream and HTTP streaming.

9. The method of claim 6, wherein the indication of the aspect sub-sampling ratio comprises an indication that the views have an asymmetric spatial resolution.

10. The method of claim 1, wherein providing the manifest file comprises sending the information to the client device in accordance with Dynamic Adaptive Streaming over HTTP (DASH).

11. An apparatus for providing video data, the apparatus comprising a server device comprising:

a memory configured to store video data; and

a hardware-based processing unit including logic circuitry, the processing unit configured to;

receive a packed frame of a representation of a plurality of representations of the video data, wherein the packed frame comprises two frames corresponding to different views of a scene for three-dimensional playback, wherein the representations act as alternatives to each other and have different coding or rendering characteristics from each other, wherein each of the representations corresponds to a separate, respective bitstream, and wherein each of the representations includes a respective plurality of segments including coded video data, the segments comprising respective independently retrievable files,

provide a manifest file to a client device, wherein the manifest file indicates a frame packing arrangement for the packed frame, wherein the manifest file includes descriptive data for each of the representations, and wherein the manifest file is separate from the segments of the representations such that the manifest file does not include any coded video data,

receive a request from the client device for at least a portion of one of the segments of one of the representations described by the manifest file, and

in response to the request, send at least the portion of the one of the segments to the client device.

12. The apparatus of claim 11, wherein the processing unit is configured to provide the manifest file in response to a request from the client device for the manifest file.

13. The apparatus of claim 11, wherein to provide the manifest file, the processing unit is configured to form a media presentation description data structure that describes three-dimensional video characteristics of each of the plurality of representations, and send the media presentation description data structure to the client device.

14. The apparatus of claim 11, wherein the processing unit is further configured to receive a request for data from a representation of multimedia content, the representation comprising the packed frame, from the client device when the client device is configured to decode packed frames having the frame packing arrangement of the packed frame, and, in response to the request, send the data to the client device.

15. The apparatus of claim 11, wherein the processing unit is configured to send information at a codec layer for a representation of multimedia content, the representation comprising the packed frame, to the client device, wherein the information comprises a frame packing arrangement supplemental enhancement information (SEI) message, and wherein the frame packing arrangement SEI message is separate from the manifest file.

16. The apparatus of claim 11, wherein the apparatus comprises at least one of:

an integrated circuit;

a microprocessor; and

a wireless communication device that includes the processing unit.

17. An apparatus for providing video data, the apparatus comprising a server device comprising:

means for receiving a packed frame of a representation of a plurality of representations of video data of multimedia content, wherein the packed frame comprises two frames corresponding to different views of a scene for three-dimensional playback, wherein the representations act as alternatives to each other and have different coding or rendering characteristics from each other, wherein each of the representations corresponds to a separate, respective bitstream, and wherein each of the representations includes a respective plurality of segments including coded video data, the segments comprising respective independently retrievable files;

means for providing a manifest file to a client device separate from the server device, wherein the manifest file indicates a frame packing arrangement for the packed frame, wherein the manifest file includes descriptive data for each of the representations, and wherein the manifest file is separate from the segments of the representations such that the manifest file does not include any coded video data;

means for receiving a request from the client device for at least a portion of one of the segments of one of the representations described by the manifest file; and

means for sending at least the portion of the one of the segments to the client device in response to the request.

18. The apparatus of claim 17, wherein the means for providing the manifest file comprises means for providing the manifest file in response to a request from the client device for representation layer data.

19. The apparatus of claim 17, wherein the means for providing the manifest file comprises:

means for forming a media presentation description data structure that describes three-dimensional video characteristics of each of the plurality of representations; and

means for sending the media presentation description data structure to the client device.

20. The apparatus of claim 17, further comprising:

means for receiving a request for data from a representation of multimedia content, the representation comprising the packed frame, from the client device when the client device is configured to decode packed frames having the frame packing arrangement of the packed frame; and

means for sending the data to the client device in response to the request.

21. The apparatus of claim 17, further comprising means for sending information at a codec layer for a representation of multimedia content, the representation comprising the packed frame, to the client device, wherein the information comprises a frame packing arrangement supplemental enhancement information (SEI) message, and wherein the frame packing arrangement SEI message is separate from the manifest file.

22. A computer program product comprising a non-transitory computer-readable storage medium having stored thereon instructions that, when executed, cause a processor of a server device for providing video data to:

receive a packed frame of a representation of a plurality of representations of video data of multimedia content, wherein the packed frame comprises two frames corresponding to different views of a scene for three-dimensional playback, wherein the representations act as alternatives to each other and have different coding or rendering characteristics from each other, wherein each of the representations corresponds to a separate, respective bitstream, and wherein each of the representations includes a respective plurality of segments including coded video data, the segments comprising respective independently retrievable files;

provide a manifest file to a client device separate from the server device, wherein the manifest file indicates a frame packing arrangement for the packed frame, wherein the manifest file includes descriptive data for each of the representations, and wherein the manifest file is separate from the segments of the representations such that the manifest file does not include any coded video data;

receive a request from the client device for at least a portion of one of the segments of one of the representations described by the manifest file; and

in response to the request, send at least the portion of the one of the segments to the client device.

23. The computer program product of claim 22, wherein the instructions that cause the processor to provide the manifest file comprise instructions that cause the processor to provide the manifest file in response to a request from the client device for representation layer data.

24. The computer program product of claim 22, wherein the instructions that cause the processor to provide the manifest file comprise instructions that cause the processor to:

form a media presentation description data structure that describes three-dimensional video characteristics of each of the plurality of representations; and

send the media presentation description data structure to the client device.

25. The computer program product of claim 22, further comprising instructions that cause the processor to:

receive a request for data from a representation of multimedia content, the representation comprising the packed frame, from the client device when the client device is configured to decode packed frames having the frame packing arrangement of the packed frame; and

in response to the request, send the data to the client device.

26. The computer program product of claim 22, further comprising instructions that cause the processor to send information at a codec layer for a representation of multimedia content, the representation comprising the packed frame, to the client device, wherein the information comprises a frame packing arrangement supplemental enhancement information (SEI) message, and wherein the frame packing arrangement SEI message is separate from the manifest file provided at the representation layer.