Hint based media content streaming

• US 8,166,191 B1
• Filed: 08/17/2009
• Issued: 04/24/2012
• Est. Priority Date: 08/17/2009
• Status: Active Grant

• Current Assignee: Adobe Inc.
• Original Assignee: Adobe Systems Incorporated (Adobe Inc.)
• Inventors: Swaminathan, Viswanathan, Kamel, Amine El
• Primary Examiner(s): Gillis, Brian J
• Application Number:
  US12/542,582
• Time in Patent Office: 981 Days
• Field of Search: 709/230, 709/231, 709/232, 709/233, 380/212
• US Class Current: 709/231
• CPC Class Codes: H04L 65/612 for unicast
• CPC Class Codes: H04L 65/65 Network streaming protocols, e.g. real-time transport protocol [RTP] or real-time control protocol [RTCP]
• CPC Class Codes: H04L 65/70 Media network packetisation
• CPC Class Codes: H04N 21/6437 Real-time Transport Protocol [RTP]
• CPC Class Codes: H04N 21/8455 involving pointers to the content, e.g. pointers to the I-frames of the video stream

Abstract

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for media content streaming can include communicating with a remote device to stream media content to the remote device, accessing hint information associated with the media content, and transmitting a data stream to the remote device based on the hint information. The hint information can include message headers arranged in a transmission order sequence that correspond to respective media data samples. Accessing hint information can include accessing a hint segment that includes the message headers interleaved with the media data samples.

Claims

1. A method performed by data processing apparatus, comprising:

communicating with a remote device to stream media content to the remote device;

accessing hint information associated with the media content, the hint information comprising message headers, arranged in a transmission order sequence, that correspond to respective media data samples, the message headers being interleaved with the media data samples on a track, the media data samples including one or more video samples and one or more audio samples, wherein separate ones of the media data samples are arranged as message payload data for corresponding ones of the message headers, wherein separate ones of the message headers comprise a message stream identifier, a message type identifier, a timestamp, and a message length value; and

transmitting a data stream to the remote device, the data stream comprising messages arranged in the transmission order sequence, wherein separate ones of the messages are based on a different one of the message headers and a corresponding different one of the media data samples,

wherein accessing hint information comprises accessing a document that comprises a hint segment, a first media metadata track, a second media metadata track, and a hint metadata track,

wherein the hint segment comprises the message headers and the media data samples,

wherein the hint metadata track includes pointers to respective ones of the message headers,

wherein the first media metadata track includes, in lieu of samples, one or more pointers to corresponding one or more audio samples of the media data samples within the hint segment, and

wherein the second media metadata track includes, in lieu of samples, one or more pointers to corresponding one or more video samples of the media data samples within the hint segment.

2. The method of claim 1, wherein transmitting the data stream to the remote device comprises transmitting the data stream in accordance with a Real-Time Messaging Protocol (RTMP), wherein each message header is arranged according to a RTMP message header format.

3. The method of claim 1, further comprising:

operating a server to deliver multiple streams to remote devices based on information including the hint information.

4. A method performed by data processing apparatus, comprising:

accessing a document that comprises hint information, a first media metadata track, and a second media metadata track, the hint information comprising a hint segment and a hint metadata track, the hint information comprising message headers, arranged in a transmission order sequence, that correspond to respective media data samples, the message headers being interleaved with the media data samples on a track, the media data samples including one or more video samples and one or more audio samples, wherein separate ones of the media data samples are arranged as message payload data for corresponding ones of the message headers, wherein separate ones of the message headers comprise a message stream identifier, a message type identifier, a timestamp, and a message length value; and

using the hint information to render media content,

wherein the hint segment comprises the message headers and the media data samples,

wherein the hint metadata track includes pointers to respective ones of the message headers,

wherein the first media metadata track includes, in lieu of samples, one or more pointers to corresponding one or more audio samples of the media data samples within the hint segment, and

wherein the second media metadata track includes, in lieu of samples, one or more pointers to corresponding one or more video samples of the media data samples within the hint segment.

5. The method of claim 4, wherein each message header is arranged according to a Real-Time Messaging Protocol (RTMP) message header format.

6. A non-transitory computer storage medium encoded with a computer program, the program comprising instructions that when executed by data processing apparatus cause the data processing apparatus to perform operations comprising:

communicating with a remote device to stream media content to the remote device;

accessing hint information associated with the media content, the hint information comprising message headers, arranged in a transmission order sequence, that correspond to respective media data samples, the message headers being interleaved with the media data samples on a track, the media data samples including one or more video samples and one or more audio samples, wherein separate ones of the media data samples are arranged as message payload data for corresponding ones of the message headers, wherein separate ones of the message headers comprise a message stream identifier, a message type identifier, a timestamp, and a message length value; and

transmitting a data stream to the remote device, the data stream comprising messages arranged in the transmission order sequence, wherein separate ones of the messages are based on a different one of the message headers and a corresponding different one of the media data samples,

wherein accessing hint information comprises accessing a document that comprises a hint segment, a first media metadata track, a second media metadata track, and a hint metadata track,

wherein the hint segment comprises the message headers and the media data samples,

wherein the hint metadata track includes pointers to respective ones of the message headers,

wherein the first media metadata track includes, in lieu of samples, one or more pointers to corresponding one or more audio samples of the media data samples within the hint segment, and

wherein the second media metadata track includes, in lieu of samples, one or more pointers to corresponding one or more video samples of the media data samples within the hint segment.

7. The non-transitory computer storage medium of claim 6, wherein transmitting the data stream to the remote device comprises transmitting the data stream in accordance with a Real-Time Messaging Protocol (RTMP), wherein each message header is arranged according to a RTMP message header format.

8. The non-transitory computer storage medium of claim 6, the operations further comprising:

operating a server to deliver multiple streams to remote devices based on information including the hint information.

9. A system comprising:

a storage device configured to store information comprising a document that includes hint information associated with media content; and

an endpoint operable to communicate with the storage device, the endpoint configured to perform operations comprising;

accessing the hint information, the hint information comprising message headers, arranged in a transmission order sequence, that correspond to respective media data samples, the message headers being interleaved with the media data samples on a track, the media data samples including one or more video samples and one or more audio samples, wherein separate ones of the media data samples are arranged as message payload data for corresponding ones of the message headers, wherein separate ones of the message headers comprise a message stream identifier, a message type identifier, a timestamp, and a message length value; and

providing a data stream based on the hint information, the data stream comprising messages arranged in the transmission order sequence, wherein separate ones of the messages are based on a different one of the message headers and a corresponding different one of the media data samples,

wherein the document comprises a hint segment, a first media metadata track, a second media metadata track, and a hint metadata track,

wherein the hint segment comprises the message headers and the media data samples,

wherein the hint metadata track includes pointers to respective ones of the message headers,

wherein the first media metadata track includes, in lieu of samples, one or more pointers to corresponding one or more audio samples of the media data samples within the hint segment, and

wherein the second media metadata track includes, in lieu of samples, one or more pointers to corresponding one or more video samples of the media data samples within the hint segment.

10. The system of claim 9, wherein providing the data stream comprises transmitting the data stream to a remote device, wherein transmitting the data stream to the remote device comprises transmitting the data stream in accordance with a Real-Time Messaging Protocol (RTMP), wherein each message header is arranged according to a RTMP message header format.