Hint based media content streaming

• US 9,282,382 B2
• Filed: 04/23/2012
• Issued: 03/08/2016
• Est. Priority Date: 08/17/2009
• Status: Active Grant

• Current Assignee: Adobe Inc.
• Original Assignee: Adobe Systems Incorporated (Adobe Inc.)
• Inventors: Swaminathan, Viswanathan, Kamel, Amine El
• Primary Examiner(s): Srivastava, Vivek
• Assistant Examiner(s): Abedin, Normin
• Application Number:
  US13/453,870
• Publication Number:
  US 20140240594A1
• Time in Patent Office: 1,415 Days
• Field of Search: 709/219, 709/220, 709/224
• US Class Current: 1/1
• CPC Class Codes: H04L 65/612 for unicast
• CPC Class Codes: H04L 65/65 Network streaming protocols, e.g. real-time transport protocol [RTP] or real-time control protocol [RTCP]
• CPC Class Codes: H04L 65/70 Media network packetisation
• CPC Class Codes: H04N 21/6437 Real-time Transport Protocol [RTP]
• CPC Class Codes: H04N 21/8455 involving pointers to the content, e.g. pointers to the I-frames of the video stream

Abstract

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for media content streaming can include communicating with a remote device to stream media content to the remote device, accessing hint information associated with the media content, and transmitting a data stream to the remote device based on the hint information. The hint information can include message headers arranged in a transmission order sequence that correspond to respective media data samples. Accessing hint information can include accessing a hint segment that includes the message headers interleaved with the media data samples.

Claims

1. A method performed by data processing apparatus, comprising:

accessing a document that comprises a hint segment, a media metadata track, and a hint metadata track,

wherein the hint segment comprises first and second message headers that correspond to respective first and second media data samples, wherein the first media data sample is associated with audio data and is interleaved with the second media data sample associated with video data, wherein the message headers are arranged within the hint segment in a transmission order sequence, wherein separate ones of the media data samples are arranged as message payload data for corresponding ones of the message headers, wherein each of the message headers comprise a message stream identifier, a message type identifier, a timestamp, and a message length value,

wherein the hint metadata track includes pointers to respective ones of the message headers,

wherein the media metadata track includes pointers to respective associated ones of the media data samples; and

generating a data stream to stream media content of the document, the data stream comprising messages arranged in the transmission order sequence, wherein each of the messages are based on a different one of the message headers and a corresponding different one of the media data samples.

2. The method of claim 1, wherein the media metadata track is a first media metadata track, wherein the document includes a second media metadata track, wherein the first media metadata track includes, in lieu of samples, one or more pointers to corresponding one or more first samples of the media data samples within the hint segment, and wherein the second media metadata track includes, in lieu of samples, one or more pointers to corresponding one or more second samples of the media data samples within the hint segment, wherein the one or more second samples are different from the one or more first samples.

3. The method of claim 1, further comprising:

transmitting the data stream to a remote device.

4. The method of claim 3, wherein transmitting the data stream to the remote device comprises transmitting the data stream in accordance with a Real-Time Messaging Protocol (RTMP), wherein each message header is arranged according to a RTMP message header format.

5. The method of claim 3, wherein transmitting the data stream to the remote device comprises multiplexing first messages that include audio data with second messages that include video data on the data stream.

6. The method of claim 1, further comprising:

operating a server to deliver multiple streams to remote devices based on the hint segment.

7. A non-transitory computer storage medium encoded with a computer program, the program comprising instructions that when executed by data processing apparatus cause the data processing apparatus to perform operations comprising:

accessing a document that comprises a hint segment, a media metadata track, and a hint metadata track,

wherein the hint segment comprises first and second message headers that correspond to respective first and second media data samples, wherein the first media data sample is associated with audio data and is interleaved with the second media data sample associated with video data wherein the message headers are arranged within the hint segment in a transmission order sequence, wherein separate ones of the media data samples are arranged as message payload data for corresponding ones of the message headers, wherein each of the message headers comprise a message stream identifier, a message type identifier, a timestamp, and a message length value,

wherein the hint metadata track includes pointers to respective ones of the message headers,

wherein the media metadata track includes pointers to respective associated ones of the media data samples; and

generating a data stream to stream media content of the document, the data stream comprising messages arranged in the transmission order sequence, wherein each of the messages are based on a different one of the message headers and a corresponding different one of the media data samples.

8. The computer storage medium of claim 7, wherein the media metadata track is a first media metadata track, wherein the document includes a second media metadata track, wherein the first media metadata track includes, in lieu of samples, one or more pointers to corresponding one or more first samples of the media data samples within the hint segment, and wherein the second media metadata track includes, in lieu of samples, one or more pointers to corresponding one or more second samples of the media data samples within the hint segment, wherein the one or more second samples are different from the one or more first samples.

9. The computer storage medium of claim 7, the operations further comprising:

transmitting the data stream to a remote device.

10. The computer storage medium of claim 9, wherein transmitting the data stream to the remote device comprises transmitting the data stream in accordance with a Real-Time Messaging Protocol (RTMP), wherein each message header is arranged according to a RTMP message header format.

11. The computer storage medium of claim 9, wherein transmitting the data stream to the remote device comprises multiplexing first messages that include audio data with second messages that include video data on the data stream.

12. A system comprising:

a computer storage device configured to store information comprising a document that contains a hint segment, a media metadata track, and a hint metadata track,

wherein the hint segment comprises first and second message headers that correspond to respective first and second media data samples, wherein the first media data sample is associated with audio data and is interleaved with the second media data sample associated with video data wherein the message headers are arranged within the hint segment in a transmission order sequence, wherein separate ones of the media data samples are arranged as message payload data for corresponding ones of the message headers, wherein each of the message headers comprise a message stream identifier, a message type identifier, a timestamp, and a message length value,

wherein the hint metadata track includes pointers to respective ones of the message headers,

wherein the media metadata track includes pointers to respective associated ones of the media data samples; and

a data processing apparatus configured to perform operations comprising (i) accessing the document, and (ii) generating a data stream to stream media content of the document, the data stream comprising messages arranged in the transmission order sequence, wherein each of the messages are based on a different one of the message headers and a corresponding different one of the media data samples.

13. The system of claim 12, wherein the media metadata track is a first media metadata track, wherein the document includes a second media metadata track, wherein the first media metadata track includes, in lieu of samples, one or more pointers to corresponding one or more first samples of the media data samples within the hint segment, and wherein the second media metadata track includes, in lieu of samples, one or more pointers to corresponding one or more second samples of the media data samples within the hint segment, wherein the one or more second samples are different from the one or more first samples.

14. The system of claim 12, the operations further comprising:

transmitting the data stream to a remote device.

15. The system of claim 14, wherein transmitting the data stream to the remote device comprises transmitting the data stream in accordance with a Real-Time Messaging Protocol (RTMP), wherein each message header is arranged according to a RTMP message header format.

16. The system of claim 14, wherein transmitting the data stream to the remote device comprises multiplexing first messages that include audio data with second messages that include video data on the data stream.

17. The system of claim 12, wherein the data processing apparatus includes the computer storage device.