ARRANGING SUB-TRACK FRAGMENTS FOR STREAMING VIDEO DATA

• US 20120023250A1
• Filed: 07/20/2010
• Published: 01/26/2012
• Est. Priority Date: 07/20/2010
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Chen, Ying, Karczewicz, Marta
• Granted Patent: US 8,930,562 B2
• Time in Patent Office: Days
• US Class Current: 709/231
• CPC Class Codes: H04L 65/612 for unicast
• CPC Class Codes: H04L 65/70 Media network packetisation
• CPC Class Codes: H04L 65/762 at the source reformatting of additional data in video distribution servers H04N21/2355
• CPC Class Codes: H04L 65/764 at the destination reformatting of additional data in video clients H04N21/4355
• CPC Class Codes: H04L 67/04 specially adapted for terminals or networks with limited capabilities; specially adapted for terminal portability
• CPC Class Codes: H04N 21/234327 by decomposing into layers, e.g. base layer and one or more enhancement layers
• CPC Class Codes: H04N 21/23608 Remultiplexing multiplex streams, e.g. involving modifying time stamps or remapping the packet identifiers
• CPC Class Codes: H04N 21/4424 Monitoring of the internal components or processes of the client device, e.g. CPU or memory load, processing speed, timer, counter or percentage of the hard disk space used error monitoring in general G06F11/30; arrangements for monitoring conditions of receiving stations in broadcast systems H04H60/32; diagnosis, testing or measuring for television receivers H04N17/04
• CPC Class Codes: H04N 21/462 Content or additional data management, e.g. creating a master electronic program guide from data received from the Internet and a Head-end, controlling the complexity of a video stream by scaling the resolution or bit-rate based on the client capabilities
• CPC Class Codes: H04N 21/8456 by decomposing the content in the time domain, e.g. in time segments

Abstract

A video file may include movie fragments divided into sub-track fragments that store all pictures of common hierarchical levels for the respective hierarchical levels. In one example, an apparatus includes an interface configured to output data according to a streaming protocol, and a control unit configured to assemble encoded video data into a plurality of sub-track fragments, each of the sub-track fragments comprising a plurality of hierarchically related video pictures of the encoded video data, wherein the plurality of hierarchically related video pictures each correspond to a common hierarchical layer receiving a request in accordance with the streaming protocol, wherein the request specifies at least one of the plurality of sub-track fragments, and, in response to the request, cause the interface to output the plurality of hierarchically related video pictures of the at least one of the plurality of sub-track fragments.

Claims

1. A method of outputting encoded video data, the method comprising:

assembling encoded video data into a plurality of sub-track fragments, each of the sub-track fragments comprising a plurality of hierarchically related coded video pictures of the encoded video data, wherein the hierarchically related coded video pictures each correspond to a common hierarchical layer;

receiving a request in accordance with a streaming protocol, wherein the request specifies at least one of the sub-track fragments; and

in response to the request, outputting the hierarchically related coded video pictures of the at least one of the plurality of sub-track fragments.

2. The method of claim 1, wherein the request comprises a hypertext transfer protocol (HTTP) partial GET request that specifies a byte range corresponding to the at least one of the plurality of sub-track fragments.

3. The method of claim 1, further comprising signaling reordering information that indicates how to reorder the coded video pictures of more than one of the sub-track fragments into a decoding order.

4. The method of claim 3, wherein signaling reordering information comprises:

producing a reassembler object that references a coded video picture of a first one of the plurality of sub-track fragments; and

storing the reassembler object in a second one of the plurality of sub-track fragments.

5. The method of claim 4, wherein producing the reassembler object comprises producing the reassembler object to include an index value for the first one of the plurality of sub-track fragments and a position value that indicates a position of the sample in the first one of the plurality of sub-track fragments.

6. The method of claim 3,

wherein signaling reordering information comprises signaling, for each sub-track fragment, a sub-track fragment identifier and a number of pictures in the sub-track fragment, the method further comprising:

arranging the sub-track fragments such that the sub-track fragments follow a fixed pattern.

7. The method of claim 1, further comprising producing a sub-track fragment header that specifies a byte range for at least one of the plurality of sub-track fragments.

8. The method of claim 7, further comprising outputting the sub-track fragment header before outputting the plurality of hierarchically related video pictures of the at least one of the plurality of sub-track fragments.

9. The method of claim 1, further comprising defining, in an initialization fragment of a media file including the sub-track fragments, at least one operation point corresponding to a subset of the plurality of sub-track fragments.

10. An apparatus for outputting encoded video data, the apparatus comprising:

an interface configured to output data according to a streaming protocol; and

a control unit configured to assemble encoded video data into a plurality of sub-track fragments, each of the sub-track fragments comprising a plurality of hierarchically related video pictures of the encoded video data, wherein the hierarchically related video pictures each correspond to a common hierarchical layer, receive a request in accordance with the streaming protocol, wherein the request specifies at least one of the plurality of sub-track fragments, and, in response to the request, cause the interface to output the hierarchically related video pictures of the at least one of the sub-track fragments.

11. The apparatus of claim 10, wherein the request comprises a hypertext transfer protocol (HTTP) partial GET request that specifies a byte range corresponding to the at least one of the plurality of sub-track fragments.

12. The apparatus of claim 10, wherein the control unit is further configured to produce a reassembler object that references a sample of a first one of the plurality of sub-track fragments, and store the reassembler object in a second one of the plurality of sub-track fragments.

13. The apparatus of claim 12, wherein the control unit is configured to produce the reassembler object to include an index value for the first one of the plurality of sub-track fragments and a position value that indicates a position of the sample in the first one of the plurality of sub-track fragments.

14. The apparatus of claim 10, wherein the control unit is configured to produce a de-multiplexing header that specifies a byte range for at least one of the plurality of sub-track fragments.

15. The apparatus of claim 14, wherein the control unit is configured to output the de-multiplexing header before outputting the plurality of hierarchically related video pictures of the at least one of the plurality of sub-track fragments.

16. The apparatus of claim 10, wherein the control unit is configured to define, in a movie box of a video file including the sub-track fragments, at least one operation point corresponding to a subset of the plurality of sub-track fragments.

17. The apparatus of claim 10, wherein the apparatus comprises at least one of:

an integrated circuit;

a microprocessor; and

a wireless communication device that includes the control unit.

18. An apparatus for outputting encoded video data, the apparatus comprising:

means for assembling encoded video data into a plurality of sub-track fragments, each of the sub-track fragments comprising a plurality of hierarchically related video pictures of the encoded video data, wherein the plurality of hierarchically related video pictures each correspond to a common hierarchical layer;

means for receiving a request in accordance with a streaming protocol, wherein the request specifies at least one of the plurality of sub-track fragments; and

means for outputting the plurality of hierarchically related video pictures of the at least one of the plurality of sub-track fragments in response to the request.

19. The apparatus of claim 18, wherein the request comprises a hypertext transfer protocol (HTTP) partial GET request that specifies a byte range corresponding to the at least one of the plurality of sub-track fragments.

20. The apparatus of claim 18, further comprising:

means for producing a reassembler object that references a sample of a first one of the plurality of sub-track fragments; and

means for storing the reassembler object in a second one of the plurality of sub-track fragments.

21. The apparatus of claim 20, wherein the means for producing the reassembler object comprises means for producing the reassembler object to include an index value for the first one of the plurality of sub-track fragments and a position value that indicates a position of the sample in the first one of the plurality of sub-track fragments.

22. The apparatus of claim 18, further comprising means for producing a de-multiplexing header that specifies a byte range for at least one of the plurality of sub-track fragments.

23. The apparatus of claim 22, further comprising means for outputting the de-multiplexing header before outputting the plurality of hierarchically related video pictures of the at least one of the plurality of sub-track fragments.

24. The apparatus of claim 18, further comprising means for defining, in a movie box of a video file including the sub-track fragments, at least one operation point corresponding to a subset of the plurality of sub-track fragments.

25. A computer program product comprising a computer-readable storage medium having stored thereon instructions that, when executed, cause a processor of a source device for outputting encoded video data to:

assemble encoded video data into a plurality of sub-track fragments, each of the sub-track fragments comprising a plurality of hierarchically related video pictures of the encoded video data, wherein the plurality of hierarchically related video pictures each correspond to a common hierarchical layer;

receive a request in accordance with a streaming protocol, wherein the request specifies at least one of the plurality of sub-track fragments; and

in response to the request, output the plurality of hierarchically related video pictures of the at least one of the plurality of sub-track fragments.

26. The computer program product of claim 25, wherein the request comprises a hypertext transfer protocol (HTTP) partial GET request that specifies a byte range corresponding to the at least one of the plurality of sub-track fragments.

27. The computer program product of claim 25, further comprising instructions to:

produce a reassembler object that references a sample of a first one of the plurality of sub-track fragments; and

store the reassembler object in a second one of the plurality of sub-track fragments.

28. The computer program product of claim 27, wherein the instructions to produce the reassembler object comprise instructions to produce the reassembler object to include an index value for the first one of the plurality of sub-track fragments and a position value that indicates a position of the sample in the first one of the plurality of sub-track fragments.

29. The computer program product of claim 25, further comprising instructions to produce a de-multiplexing header that specifies a byte range for at least one of the plurality of sub-track fragments.

30. The computer program product of claim 29, further comprising instructions to output the de-multiplexing header before outputting the plurality of hierarchically related video pictures of the at least one of the plurality of sub-track fragments.

31. The computer program product of claim 25, further comprising instructions to define, in a movie box of a video file including the sub-track fragments, at least one operation point corresponding to a subset of the plurality of sub-track fragments.

32. A method of receiving encoded video data, the method comprising:

receiving information from a source device that describes hierarchical levels of video data for a movie fragment;

determining a subset of the hierarchical levels of video data to request;

for each of the hierarchical levels of the subset, sending no more than one request to the source device to retrieve all of the video data of the movie fragment at the hierarchical level;

receiving the video data of the determined subset of the hierarchical levels; and

decoding and displaying the received video data.

33. The method of claim 32, wherein the information that describes the hierarchical levels comprises a de-multiplexing header that specifies, for each of the hierarchical levels, a byte range in the movie fragment of the video data corresponding to the hierarchical level.

34. The method of claim 32, wherein the received video data comprises a first sub-track fragment for a first hierarchical level and a second sub-track fragment for a second hierarchical level, wherein the first sub-track fragment includes a plurality of video pictures, and wherein the second sub-track fragment includes a reassembler object that references one of the plurality of video pictures of the first sub-track fragment, the method further comprising:

arranging a sequence of video pictures from the first sub-track fragment and the second sub-track fragment in decoding order using the reassembler object.

35. The method of claim 32, wherein the information that describes the hierarchical levels comprises a movie box of a video file including the video data, wherein the movie box defines an operation point corresponding to the subset of the hierarchical levels, and wherein determining the subset comprises selecting the operation point.

36. An apparatus for receiving encoded video data, the apparatus comprising:

an interface configured to receive information from a source device that describes hierarchical levels of video data for a movie fragment; and

a control unit configured to determine a subset of the hierarchical levels of video data to request, wherein, for each of the hierarchical levels of the subset, the control unit is configured to send no more than one request to the source device to retrieve all of the video data of the movie fragment at the hierarchical level, wherein the interface is configured to receive the video data of the determined subset of the hierarchical levels in response to the requests.

37. The apparatus of claim 36, wherein the information that describes the hierarchical levels comprises a de-multiplexing header that specifies, for each of the hierarchical levels, a byte range in the movie fragment of the video data corresponding to the hierarchical level.

38. The apparatus of claim 36, wherein the received video data comprises a first sub-track fragment for a first hierarchical level and a second sub-track fragment for a second hierarchical level, wherein the first sub-track fragment includes a plurality of video pictures, and wherein the second sub-track fragment includes a reassembler object that references one of the plurality of video pictures of the first sub-track fragment, and wherein the control unit is configured to arrange a sequence of video pictures from the first sub-track fragment and the second sub-track fragment in decoding order using the reassembler object.

39. The apparatus of claim 36, wherein the information that describes the hierarchical levels comprises a movie box of a video file including the video data, wherein the movie box defines an operation point corresponding to the subset of the hierarchical levels, and wherein to determine the subset, the control unit is configured to select the operation point.

40. The apparatus of claim 36, wherein the apparatus comprises at least one of:

an integrated circuit;

a microprocessor; and

a wireless communication device that includes the control unit.

41. An apparatus for receiving encoded video data, the apparatus comprising:

means for receiving information from a source device that describes hierarchical levels of video data for a movie fragment;

means for determining a subset of the hierarchical levels of video data to request;

means for sending, for each of the hierarchical levels of the subset, no more than one request to the source device to retrieve all of the video data of the movie fragment at the hierarchical level;

means for receiving the video data of the determined subset of the hierarchical levels; and

means for decoding and displaying the received video data.

42. The apparatus of claim 41, wherein the information that describes the hierarchical levels comprises a de-multiplexing header that specifies, for each of the hierarchical levels, a byte range in the movie fragment of the video data corresponding to the hierarchical level.

43. The apparatus of claim 41, wherein the received video data comprises a first sub-track fragment for a first hierarchical level and a second sub-track fragment for a second hierarchical level, wherein the first sub-track fragment includes a plurality of video pictures, and wherein the second sub-track fragment includes a reassembler object that references one of the plurality of video pictures of the first sub-track fragment, the apparatus further comprising:

means for arranging a sequence of video pictures from the first sub-track fragment and the second sub-track fragment in decoding order using the reassembler object.

44. The apparatus of claim 41, wherein the information that describes the hierarchical levels comprises a movie box of a video file including the video data, wherein the movie box defines an operation point corresponding to the subset of the hierarchical levels, and wherein the means for determining the subset comprises means for selecting the operation point.

45. A computer program product comprising a computer-readable storage medium having stored thereon instructions that, when executed, cause a processor of a device for receiving encoded video data to:

receive information from a source device that describes hierarchical levels of video data for a movie fragment;

determine a subset of the hierarchical levels of video data to request;

for each of the hierarchical levels of the subset, send no more than one request to the source device to retrieve all of the video data of the movie fragment at the hierarchical level;

receive the video data of the determined subset of the hierarchical levels; and

decode and display the received video data.

46. The computer program product of claim 45, wherein the information that describes the hierarchical levels comprises a de-multiplexing header that specifies, for each of the hierarchical levels, a byte range in the movie fragment of the video data corresponding to the hierarchical level.

47. The computer program product of claim 45, wherein the received video data comprises a first sub-track fragment for a first hierarchical level and a second sub-track fragment for a second hierarchical level, wherein the first sub-track fragment includes a plurality of video pictures, and wherein the second sub-track fragment includes a reassembler object that references one of the plurality of video pictures of the first sub-track fragment, the computer-readable storage medium further comprising instructions to:

arrange a sequence of video pictures from the first sub-track fragment and the second sub-track fragment in decoding order using the reassembler object.

48. The computer program product of claim 45, wherein the information that describes the hierarchical levels comprises a movie box of a video file including the video data, wherein the movie box defines an operation point corresponding to the subset of the hierarchical levels, and wherein the instructions to determine the subset comprise instructions to select the operation point.