Signaling data for multiplexing video components

• US 9,185,439 B2
• Filed: 01/06/2011
• Issued: 11/10/2015
• Est. Priority Date: 07/15/2010
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Chen, Ying, Karczewicz, Marta, Wang, Yong
• Primary Examiner(s): Bui, Kieu Oanh T
• Application Number:
  US12/986,028
• Publication Number:
  US 20120013746A1
• Time in Patent Office: 1,769 Days
• Field of Search: 725/87, 725/92, 725/93
• US Class Current: 1/1
• CPC Class Codes: G11B 27/034 on discs G11B27/036, G11B27/038 take precedence
• CPC Class Codes: G11B 27/3027 used signal is digitally coded
• CPC Class Codes: H04N 21/235 Processing of additional data, e.g. scrambling of additional data or processing content descriptors
• CPC Class Codes: H04N 21/435 Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream
• CPC Class Codes: H04N 21/4621 Controlling the complexity of the content stream or additional data, e.g. lowering the resolution or bit-rate of the video stream for a mobile client with a small screen arrangements for using the results of monitoring on user's side in broadcast systems H04H60/65; flow control in packet networks H04L47/10
• CPC Class Codes: H04N 21/8456 by decomposing the content in the time domain, e.g. in time segments
• CPC Class Codes: H04N 21/85406 involving a specific file format, e.g. MP4 format

Abstract

A server may provide information describing characteristics of audio and video components to a client, separately from encoded samples of the audio and video components themselves. The client may use the information to select components, and then to request the selected components, e.g., in accordance with a streaming network protocol. In one example, an apparatus for sending encapsulated video data includes a processor configured to determine characteristics for components of a plurality of representations of video content, wherein the characteristics comprise at least one of a frame rate, a profile indicator, a level indicator, and dependencies between the components, and one or more interfaces configured to send the characteristics to a client device, receive a request from the client device for at least one of the components after sending the characteristics, and send the requested components to the client device in response to the request.

Claims

1. A method of sending encapsulated video data, the method comprising:

sending, via a Hypertext Transfer Protocol (HTTP)-based network streaming protocol, a manifest file comprising characteristics for components of a plurality of alternative representations of video content to a client device, wherein the characteristics comprise at least one of a frame rate, a profile indicator, a level indicator, and dependencies between the components, and wherein the manifest file conforms to the HTTP-based network streaming protocol;

receiving, via the HTTP-based network streaming protocol, a request from the client device for at least one of the components after sending the manifest file; and

sending, via the HTTP-based network streaming protocol, the requested components to the client device in response to the request.

2. The method of claim 1, wherein at least two of the components are stored in separate files, and wherein the manifest file comprises characteristics for each of the at least two of the components.

3. The method of claim 1, further comprising:

storing the manifest file separately from one or more files storing encoded samples for the components,

wherein sending the manifest file comprises;

receiving a first request for the manifest file; and

sending the manifest file independently of the one or more files storing the encoded samples in response to the first request, and

wherein the request for the at least one of the video components comprises a second, different request.

4. The method of claim 1, wherein the manifest file is separate from the components, the method further comprising:

assigning an identifier to the manifest file that associates the manifest file with a multimedia content comprising the plurality of alternative representations; and

assigning unique identifiers to the representations of the multimedia content.

5. The method of claim 1, further comprising sending component identifier values for the components, wherein at least one of the component identifier values is different than a track identifier value for the component corresponding to the at least one of the component identifier values.

6. The method of claim 5, further comprising sending information indicating a correspondence between component identifier values for the components and track identifier values for the components in one or more files storing encoded samples for the components.

7. The method of claim 6, further comprising sending, for each of the components of the one or more files, information indicative of byte offsets to fragments within the component, decoding times of first samples in the fragments, random access points in the fragments, and indications of whether the fragments belong to a new segment of the component.

8. The method of claim 1, further comprising sending information indicating that a set of the components are switchable to each other, and wherein the request specifies at least one of the set of components.

9. The method of claim 1, further comprising sending information indicative of the dependencies between the components and an ordering of the dependencies between the components for a decoding order of the components in an access unit.

10. The method of claim 1, further comprising sending information indicative of the dependencies between the components and a temporal layer difference between a first component and a second component that is dependent on the first component.

11. The method of claim 1, further comprising sending information indicative of a number of target views for output for one or more of the plurality of alternative representations.

12. The method of claim 1, further comprising sending information indicative of possible multiplexing intervals for a combination of two or more of the components, and wherein the request specifies fragments of any of the two or more of the components having decoding times within a common one of the multiplexing intervals.

13. The method of claim 1, wherein the characteristics comprise a first set of characteristics, and wherein sending the manifest file comprising the characteristics comprises sending information indicative of a first temporal duration of the components to which the first set of characteristics correspond, the method further comprising sending a second set of characteristics for the components and a second temporal duration of the components to which the second set of characteristics correspond.

14. An apparatus for sending encapsulated video data, the apparatus comprising:

a processor configured to determine characteristics for components of a plurality of alternative representations of video content, wherein the characteristics comprise at least one of a frame rate, a profile indicator, a level indicator, and dependencies between the components, and to form a manifest file comprising the characteristics, and wherein the manifest file conforms to a Hypertext Transfer Protocol (HTTP)-based network streaming protocol; and

one or more interfaces configured to;

send, via the HTTP-based network streaming protocol the manifest file to a client device,

receive, via the HTTP-based network streaming protocol, a request from the client device for at least one of the components after sending the manifest file, and

send, via the HTTP-based network streaming protocol, the requested components to the client device in response to the request.

15. The apparatus of claim 14, wherein the characteristics further comprise component identifier values for the components, wherein at least one of the component identifier values is different than a track identifier value for the component corresponding to the at least one of the component identifier values, and wherein the characteristics comprise information indicating a correspondence between component identifier values for the components and track identifier values for the components in one or more files storing encoded samples for the components.

16. The apparatus of claim 15, wherein the characteristics further comprise, for each of the components of the one or more files, information indicative of byte offsets to fragments within the component, decoding times of first samples in the fragments, random access points in the fragments, and indications of whether the fragment belongs to a new segment of the component.

17. The apparatus of claim 14, wherein the characteristics comprise information indicative of the dependencies between the components and an ordering of the dependencies between the components for a decoding order of the components in an access unit.

18. The apparatus of claim 14, wherein the characteristics comprise information indicative of the dependencies between the components and a temporal layer difference between a first component and a second component that is dependent on the first component.

19. The apparatus of claim 14, wherein the characteristics comprise information indicative of a number of target views for output for one or more of the plurality of alternative representations.

20. The apparatus of claim 14, wherein the characteristics comprise information indicative of possible multiplexing intervals for a combination of two or more of the components, and wherein the request specifies fragments of any of the two or more of the components having decoding times within a common one of the multiplexing intervals.

21. The apparatus of claim 14, wherein the characteristics comprise a first set of characteristics, and wherein the one or more interfaces are configured to send information indicative of a first temporal duration of the components to which the first set of characteristics correspond, and wherein the processor is further configured to generate a second set of characteristics for the components and a second temporal duration of the components to which the second set of characteristics correspond, and wherein the one or more interfaces are configured to send the second set of characteristics.

22. The apparatus of claim 14, wherein the apparatus comprises at least one of:

an integrated circuit;

a microprocessor; and

a wireless communication device that includes the processor.

23. An apparatus for sending encapsulated video data, the apparatus comprising:

means for sending, via a Hypertext Transfer Protocol (HTTP)-based network streaming protocol, a manifest file comprising characteristics for components of a plurality of alternative representations of video content to a client device, wherein the characteristics comprise at least one of a frame rate, a profile indicator, a level indicator, and dependencies between the components, and wherein the manifest file conforms to the HTTP-based network streaming protocol;

means for receiving, via the HTTP-based network streaming protocol, a request from the client device for at least one of the components after sending the manifest file; and

means for sending, via the HTTP-based network streaming protocol, the requested components to the client device in response to the request.

24. The apparatus of claim 23, wherein means for sending the manifest file further comprises:

means for sending component identifier values for the components, wherein at least one of the component identifier values is different than a track identifier value for the component corresponding to the at least one of the component identifier values;

means for sending information indicating a correspondence between component identifier values for the components and track identifier values for the components in one or more files storing encoded samples for the components; and

means for sending, for each of the components of the one or more files, information indicative of byte offsets to fragments within the component, decoding times of first samples in the fragments, random access points in the fragments, and indications of whether the fragment belongs to a new segment of the component.

25. The apparatus of claim 23, wherein the means for sending the manifest file comprises means for sending information indicative of the dependencies between the components and an ordering of the dependencies between the components for a decoding order of the components in an access unit.

26. The apparatus of claim 23, wherein the means for sending the manifest file comprises means for sending information indicative of the dependencies between the components and a temporal layer difference between a first component and a second component that is dependent on the first component.

27. The apparatus of claim 23, wherein the means for sending the manifest file comprises means for sending information indicative of possible multiplexing intervals for a combination of two or more of the components, and wherein the request specifies fragments of any of the two or more of the components having decoding times within a common one of the multiplexing intervals.

28. The apparatus of claim 23, wherein the characteristics comprise a first set of characteristics, and wherein the means for sending the manifest file comprises means for sending information indicative of a first temporal duration of the components to which the first set of characteristics correspond, further comprising means for sending a second set of characteristics for the components and a second temporal duration of the components to which the second set of characteristics correspond.

29. A computer program product comprising a computer-readable storage medium having stored thereon instructions that, when executed, cause a processor of a source device for sending encoded video data to:

send, via a Hypertext Transfer Protocol (HTTP)-based network streaming protocol, a manifest file comprising characteristics for components of a plurality of alternative representations of video content to a client device, wherein the characteristics comprise at least one of a frame rate, a profile indicator, a level indicator, and dependencies between the components, and wherein the manifest file conforms to a Hypertext Transfer Protocol (HTTP)-based network streaming protocol;

receive, via the HTTP-based network streaming protcol, a request from the client device for at least one of the components after sending the manifest file; and

send, via the HTTP-based network streaming protcol, the requested components to the client device in response to the request.

30. The computer program product of claim 29, wherein the instructions that cause the processor to send the manifest file further comprises instructions that cause the processor to:

send component identifier values for the components, wherein at least one of the component identifier values is different than a track identifier value for the component corresponding to the at least one of the component identifier values;

send information indicating a correspondence between component identifier values for the components and track identifier values for the components in one or more files storing encoded samples for the components; and

send, for each of the components of the one or more files, information indicative of byte offsets to fragments within the component, decoding times of first samples in the fragments, random access points in the fragments, and indications of whether the fragment belongs to a new segment of the component.

31. The computer program product of claim 29, wherein the instructions that cause the processor to send the manifest file comprise instructions that cause the processor to send information indicative of the dependencies between the components, an ordering of the dependencies between the components for a decoding order of the components in an access unit, and a temporal layer difference between a first component and a second component that is dependent on the first component.

32. The computer program product of claim 29, wherein the instructions that cause the processor to send the manifest file comprise instructions that cause the processor to send information indicative of a number of target views for output for one or more of the plurality of alternative representations.

33. The computer program product of claim 29, wherein the instructions that cause the processor to send the manifest file comprise instructions that cause the processor to send information indicative of possible multiplexing intervals for a combination of two or more of the components, and wherein the request specifies fragments of any of the two or more of the components having decoding times within a common one of the multiplexing intervals.

34. The computer program product of claim 29, wherein the characteristics comprise a first set of characteristics, and further comprising instructions that cause the processor to send information indicative of a first temporal duration of the components to which the first set of characteristics correspond, and to send a second set of characteristics for the components and a second temporal duration of the components to which the second set of characteristics correspond.

35. A method of receiving encapsulated video data, the method comprising:

requesting, via a Hypertext Transfer Protocol (HTTP)-based network streaming protocol, a manifest file comprising characteristics for components of a plurality of alternative representations of video content from a source device, wherein the characteristics comprise at least one of a frame rate, a profile indicator, a level indicator, and dependencies between the components, and wherein the manifest file conforms to the HTTP-based network streaming protocol;

selecting one or more of the components based on the characteristics;

requesting, via the HTTP-based network streaming protocol, samples of the selected components; and

decoding and presenting the samples after the samples have been received.

36. The method of claim 35, further comprising:

receiving information indicating a correspondence between component identifier values for the selected components and track identifier values for the components in one or more files storing encoded samples for the components; and

receiving information indicative of byte offsets to fragments within each of the selected components, decoding times of first samples in the fragments, random access points in the fragments, and indications of whether the fragments belongs to new segments of the respective components;

wherein requesting the samples comprises requesting samples from tracks of the one or more files corresponding to the track identifier values corresponding to the component identifier values for the selected components based on the byte offsets, the decoding times, the random access points, and the indications of whether the fragments belong to new segments.

37. The method of claim 35, further comprising:

receiving information indicating that at least one of the selected components is dependent on another component; and

requesting samples of the component on which the one of the selected components is dependent.

38. The method of claim 35, wherein requesting the samples of the selected components comprises:

determining a next multiplexing interval;

determining ones of the selected components having fragments that start in the next multiplexing interval; and

requesting the fragments that start in the next multiplexing interval from the determined ones of the selected components.

39. The method of claim 35, wherein the characteristics comprise a first set of characteristics, the method further comprising:

receiving information indicative of a first temporal duration of the components to which the first set of characteristics correspond;

requesting a second set of characteristics for the components corresponding to a second temporal duration of the components to which the second set of characteristics correspond; and

requesting samples from the components corresponding to the second temporal duration based on the second set of characteristics.

40. An apparatus for receiving encapsulated video data, the apparatus comprising:

one or more interfaces configured to request, via a Hypertext Transfer Protocol (HTTP)-based network streaming protocol a manifest file comprising characteristics for components of a plurality of alternative representations of video content from a source device, wherein the characteristics comprise at least one of a frame rate, a profile indicator, a level indicator, and dependencies between the components, and wherein the manifest file conforms to a Hypertext Transfer Protocol (HTTP)-based network streaming protocol; and

a processor configured to select one or more of the components based on the characteristics, and to cause the one or more interfaces to submit requests, via the HTTP-based network streaming protocol, for samples of the selected components to the source device.

41. The apparatus of claim 40, wherein the processor is configured to receive information indicating a correspondence between component identifier values for the selected components and track identifier values for the components in one or more files storing encoded samples for the components, receive information indicative of byte offsets to fragments within each of the selected components, decoding times of first samples in the fragments, random access points in the fragments, and indications of whether the fragments belongs to new segments of the respective components, and to construct the requests for the samples from tracks of the one or more files corresponding to the track identifier values corresponding to the component identifier values for the selected components based on the byte offsets, the decoding times, the random access points, and the indications of whether the fragments belong to new segments.

42. The apparatus of claim 40, wherein the processor is configured to receive information indicating that at least one of the selected components is dependent on another component, and request samples of the component on which the one of the selected components is dependent.

43. The apparatus of claim 40, wherein to generate the requests for the samples of the selected components, the processor is configured to determine a next multiplexing interval, determine ones of the selected components having fragments that start in the next multiplexing interval, and request the fragments that start in the next multiplexing interval from the determined ones of the selected components.

44. The apparatus of claim 40, wherein the characteristics comprise a first set of characteristics, and wherein the processor is configured to receive information indicative of a first temporal duration of the components to which the first set of characteristics correspond, request a second set of characteristics for the components corresponding to a second temporal duration of the components to which the second set of characteristics correspond, and request samples from the components corresponding to the second temporal duration based on the second set of characteristics.

45. An apparatus for receiving encapsulated video data, the apparatus comprising:

means for requesting, via a Hypertext Transfer Protocol (HTTP)-based network streaming protocol, a manifest file comprising characteristics for components of a plurality of alternative representations of video content from a source device, wherein the characteristics comprise at least one of a frame rate, a profile indicator, a level indicator, and dependencies between the components, and wherein the manifest file conforms to the HTTP-based network streaming protocol;

means for selecting one or more of the components based on the characteristics;

means for requesting, via the HTTP-based network streaming protocol, samples of the selected components; and

means for decoding and presenting the samples after the samples have been received.

46. The apparatus of claim 45, further comprising:

means for receiving information indicating a correspondence between component identifier values for the selected components and track identifier values for the components in one or more files storing encoded samples for the components; and

means for receiving information indicative of byte offsets to fragments within each of the selected components, decoding times of first samples in the fragments, random access points in the fragments, and indications of whether the fragments belongs to new segments of the respective components;

wherein the means for requesting the samples comprises means for requesting samples from tracks of the one or more files corresponding to the track identifier values corresponding to the component identifier values for the selected components based on the byte offsets, the decoding times, the random access points, and the indications of whether the fragments belong to new segments.

47. The apparatus of claim 45, further comprising:

means for receiving information indicating that at least one of the selected components is dependent on another component; and

means for requesting samples of the component on which the one of the selected components is dependent.

48. The apparatus of claim 45, wherein the means for requesting the samples of the selected components comprises:

means for determining a next multiplexing interval;

means for determining ones of the selected components having fragments that start in the next multiplexing interval; and

means for requesting the fragments that start in the next multiplexing interval from the determined ones of the selected components.

49. The apparatus of claim 45, wherein the characteristics comprise a first set of characteristics, the apparatus further comprising:

means for receiving information indicative of a first temporal duration of the components to which the first set of characteristics correspond;

means for requesting a second set of characteristics for the components corresponding to a second temporal duration of the components to which the second set of characteristics correspond; and

means for requesting samples from the components corresponding to the second temporal duration based on the second set of characteristics.

50. A computer program product comprising a computer readable storage medium having stored thereon instructions that, when executed, cause a processor of a device for receiving encapsulated video data to:

request, via a Hypertext Transfer Protocol (HTTP)-based network streaming protocol, a manifest file comprising characteristics for components of a plurality of alternative representations of video content from a source device, wherein the characteristics comprise at least one of a frame rate, a profile indicator, a level indicator, and dependencies between the components, and wherein the manifest file conforms to the HTTP-based network streaming protocol;

select one or more of the components based on the characteristics;

request, via the HTTP-based network streaming protocol, samples of the selected components; and

decode and present the samples after the samples have been received.

51. The computer program product of claim 50, further comprising instructions that cause the processor to:

receive information indicating a correspondence between component identifier values for the selected components and track identifier values for the components in one or more files storing encoded samples for the components; and

receive information indicative of byte offsets to fragments within each of the selected components, decoding times of first samples in the fragments, random access points in the fragments, and indications of whether the fragments belongs to new segments of the respective components;

wherein the instructions that cause the processor to request the samples comprise instructions that cause the processor to request samples from tracks of the one or more files corresponding to the track identifier values corresponding to the component identifier values for the selected components based on the byte offsets, the decoding times, the random access points, and the indications of whether the fragments belong to new segments.

52. The computer program product of claim 50, further comprising instructions that cause the processor to:

receive information indicating that at least one of the selected components is dependent on another component; and

request samples of the component on which the one of the selected components is dependent.

53. The computer program product of claim 50, wherein the instructions that cause the processor to request the samples of the selected components comprise instructions that cause the processor to:

determine a next multiplexing interval;

determine ones of the selected components having fragments that start in the next multiplexing interval; and

request the fragments that start in the next multiplexing interval from the determined ones of the selected components.

54. The computer program product of claim 50, wherein the characteristics comprise a first set of characteristics, further comprising instructions that cause the processor to:

receive information indicative of a first temporal duration of the components to which the first set of characteristics correspond;

request a second set of characteristics for the components corresponding to a second temporal duration of the components to which the second set of characteristics correspond; and

request samples from the components corresponding to the second temporal duration based on the second set of characteristics.