Real-time priority-based media communication

• US 8,285,867 B1
• Filed: 11/24/2011
• Issued: 10/09/2012
• Est. Priority Date: 02/13/2003
• Status: Expired due to Term

• Current Assignee: Adobe Inc.
• Original Assignee: Adobe Systems Incorporated (Adobe Inc.)
• Inventors: Edelman, Bradley A., Gay, Jonathan, Lozben, Slavik, Shetty, Pritham
• Primary Examiner(s): SHAW, PELING ANDREW
• Application Number:
  US13/304,311
• Time in Patent Office: 320 Days
• Field of Search: None
• US Class Current: 709/231
• CPC Class Codes: H04L 47/10 Flow control; Congestion control
• CPC Class Codes: H04L 47/2416 Real-time traffic
• CPC Class Codes: H04L 47/2433 Allocation of priorities to traffic types
• CPC Class Codes: H04L 65/60 Network streaming of media packets
• CPC Class Codes: H04L 65/65 Network streaming protocols, e.g. real-time transport protocol [RTP] or real-time control protocol [RTCP]
• CPC Class Codes: H04L 65/762 at the source reformatting of additional data in video distribution servers H04N21/2355
• CPC Class Codes: H04N 21/2353 specifically adapted to content descriptors, e.g. coding, compressing or processing of metadata
• CPC Class Codes: H04N 21/2389 Multiplex stream processing, e.g. multiplex stream encrypting
• CPC Class Codes: H04N 21/262 Content or additional data distribution scheduling, e.g. sending additional data at off-peak times, updating software modules, calculating the carousel transmission frequency, delaying a video stream transmission, generating play-lists scheduling strategies for dispatcher in multiprogramming arrangements G06F9/4881; arrangements for scheduling broadcast services or broadcast-related services H04H60/06; flow control in packet networks H04L47/10; establishing a time schedule or organising the servicing of application requests H04L67/62
• CPC Class Codes: H04N 7/173 with two-way working, e.g. subscriber sending a programme selection signal

Abstract

Representative embodiments are disclosed of a real-time priority-based communication system and method for communicating media streams made up of multiple media message sub-streams, the communication system comprising a chunk configurator for dividing media message sub-streams into chunks, a state machine configured to translate between information regarding the media message sub-streams and the chunks and state associations to the information, the state associations written into a header for each of the chunks, a queue for holding the chunks waiting to be transmitted, and a processor for executing a scheduling algorithm, wherein the scheduling algorithm determines ones of the chunks in the queue to transmit next.

Claims

1. A non-transitory computer readable medium storing a program to cause at least one processor to perform operations comprising:

obtaining one or more streams;

processing the obtained one or more streams, wherein the processing includes

dividing each of the one or more streams into a plurality of chunks, wherein each chunk of the plurality of chunks has an associated data payload and an associated chunk control byte including a chunk type identifier, and

writing to a chunk control byte rendering information relating to a media rendering state associated with a chunk and one or more subsequent chunks of the plurality of chunks, wherein a portion of the rendering information that remains unchanged from the chunk to the one or more subsequent chunks is written to the chunk control byte of the chunk but omitted from the chunk control byte of the one or more subsequent chunks in accordance with the chunk type identifiers of the chunk and the one or more subsequent chunks, such that the chunk control bytes have variable lengths; and

outputting the processed one or more streams.

2. The computer readable medium of claim 1, wherein the writing comprises:

if the chunk is associated with a new stream from among the one or more streams, writing to the chunk control byte of the chunk

a new stream chunk type identifier as the chunk type identifier, and

rendering information relating to a media rendering state corresponding to the new stream chunk comprising a plurality of new parameters, wherein the media rendering state corresponding to the new stream chunk uses no parameters from a previous chunk of the one or more streams,

if the chunk is associated with a new media of the new stream, writing to the chunk control byte of the chunk

a new media chunk type identifier as the chunk type identifier, and

rendering information relating to a media rendering state corresponding to the new media chunk comprising a plurality of new parameters, wherein the media rendering state corresponding to the new media chunk uses at least one other parameter from a previous chunk of the new stream,

if the chunk is associated with a new time of the new stream, writing to the chunk control byte of the chunk

a new time chunk type identifier as the chunk type identifier, and

rendering information relating to a media rendering state corresponding to the new time chunk comprising at least one new parameter, wherein the media rendering state corresponding to the new time chunk uses a plurality of other parameters from a previous chunk of the new stream, and

else, writing to the chunk control byte a continuation chunk type identifier as the chunk type identifier, wherein a media rendering state corresponding to the continuation chunk uses rendering information from previous chunks.

3. The computer readable medium of claim 2, wherein the parameters include one or more of a stream ID parameter, a length parameter, a media type parameter, and a time delta parameter.

4. The computer readable medium of claim 1, the operations further comprising:

writing, to the associated chunk control byte of each of the plurality of chunks, an associated chunk identifier to indicate whether the associated data payload includes low-level protocol instructions or media content.

5. The computer readable medium of claim 1, wherein the outputting comprises:

storing the plurality of chunks of the processed one or more streams to a storage device.

6. The computer readable medium of claim 1, wherein the outputting comprises:

playing the plurality of chunks of the processed one or more streams by a media player of a computing device.

7. The computer readable medium of claim 6, wherein the playing comprises:

reading, from a chunk control byte of a chunk from among the plurality of chunks, rendering information relating to the media rendering state associated with the chunk and one or more subsequent chunks; and

controlling a media rendering state for the media player based on the information read from the chunk control byte for playing the chunk and one or more subsequent chunks.

8. The computer readable medium of claim 1, wherein the outputting comprises:

streaming the plurality of chunks of the processed one or more streams from a media streaming server computer.

9. The computer readable medium of claim 8, wherein the streaming comprises:

queuing the plurality of chunks for transmission based on a media type of each of the plurality of chunks; and

transmitting ones of the queued plurality of chunks based on a scheduling algorithm.

10. The computer readable medium of claim 9, wherein the queuing comprises:

staging the plurality of chunks according to a media type of the chunk, wherein the media type comprises one of an audio format chunk, a video format chunk, and a data format chunk.

11. The computer readable medium of claim 9, wherein the queuing comprises:

assigning a priority to each of the one or more streams;

totaling a priority for each stream;

calculating a desired transmission rate for each of the one or more streams using the assigned priority and the totaled priority;

counting a sub-total of the plurality of associated chunks transmitted per the one or more streams during a preset period;

counting a total number of the plurality of associated chunks transmitted during the preset period;

calculating an actual transmission rate for each stream of the one or more streams using the sub-total and the total; and

calculating a ratio between the desired transmission rate and the actual transmission rate.

12. A system comprising:

a storage device; and

one or more computer systems communicatively coupled with the storage device and operable to interact with a user device, the one or more computer systems being configured to perform operations including;

obtaining one or more streams;

processing the obtained one or more streams, wherein the processing includes

dividing each of the one or more streams into a plurality of chunks, wherein each chunk has an associated data payload and an associated chunk control byte including a chunk type identifier, and

writing to a chunk control byte rendering information relating to a media rendering state associated with a chunk and one or more subsequent chunks of the plurality of chunks, wherein a portion of the rendering information that remains unchanged from the chunk to the one or more subsequent chunks is written to the chunk control byte of the chunk but omitted from the chunk control byte of the subsequent one or more chunks in accordance with the chunk type identifiers of the chunk and the one or more subsequent chunks, such that the chunk control bytes have variable lengths; and

outputting the processed one or more streams.

13. The system of claim 12, wherein the writing operation further comprises operations including:

if the chunk is associated with a new stream from among the one or more streams, writing to the chunk control byte of the chunk

a new stream chunk type identifier as the chunk type identifier, and

rendering information relating to a media rendering state corresponding to the new stream chunk comprising a plurality of new parameters, wherein the media rendering state corresponding to the new stream chunk uses no parameters from a previous chunk of the one or more streams,

if the chunk is associated with a new media of the new stream, writing to the chunk control byte of the chunk

a new media chunk type identifier as the chunk type identifier, and

rendering information relating to a media rendering state corresponding to the new media chunk comprising a plurality of new parameters, wherein the media rendering state corresponding to the new media chunk uses at least one other parameter from a previous chunk of the new stream,

if the chunk is associated with a new time of the new stream, writing to the chunk control byte of the chunk

a new time chunk type identifier as the chunk type identifier, and

rendering information relating to a media rendering state corresponding to the new time chunk comprising at least one new parameter, wherein the media rendering state corresponding to the new time chunk uses a plurality of other parameters from a previous chunk of the new stream, and

else, writing to the chunk control byte a continuation chunk type identifier as the chunk type identifier, wherein a media rendering state corresponding to the continuation chunk uses rendering information from previous chunks.

14. The system of claim 13, wherein the parameters include one or more of a stream ID parameter, a length parameter, a media type parameter, and a time delta parameter.

15. The system of claim 12, wherein the one or more computer systems consist of one computer system, the user device is a user interface device, and the one computer system comprises the user interface device,

wherein the one computer system is further configured to perform operations including storing the plurality of chunks of the processed one or more streams to the storage device.

16. The system of claim 12, wherein the one or more computer systems consist of one computer system, the user device is a user interface device, and the one computer system comprises the user interface device, wherein the one computer system is further configured to perform operations including:

playing the plurality of chunks of the processed one or more streams by using a media player of the user interface device, wherein the playing operation comprises operations including;

reading, from a chunk control byte of a chunk from among the plurality of chunks, rendering information relating to the media rendering state associated with the chunk and one or more subsequent chunks; and

controlling a media rendering state for the media player based on the information read from the chunk control byte for playing the chunk and one or more subsequent chunks.

17. The system of claim 12, wherein:

the one or more computer systems comprise a media streaming server operable to interact with the user device through a data communication network, the user device is operable to interact with the media streaming server as a client, and

the media streaming server is configured to perform said outputting the processed one or more streams at least in part by streaming the plurality of chunks of the processed one or more streams from the media streaming server to the client.

18. The system of claim 17, wherein the media streaming server is further configured to perform the streaming at least in part by:

queuing the plurality of chunks for transmission based on a media type of each of the plurality of chunks; and

transmitting ones of the queued plurality of chunks based on a scheduling algorithm.

19. The system of claim 18, wherein the media streaming server is further configured to perform the queuing at least in part by staging the plurality of chunks according to a media type of the chunk, wherein the media type comprises one of an audio format chunk, a video format chunk, and a data format chunk.

20. The system of claim 18, wherein the media streaming server is further configured to perform the queuing at least in part by:

assigning a priority to each of the one or more streams;

totaling a priority for each stream;

calculating a desired transmission rate for each of the one or more streams using the assigned priority and the totaled priority;

counting a sub-total of the plurality of associated chunks transmitted per the one or more streams during a preset period;

counting a total number of the plurality of associated chunks transmitted during the preset period;

calculating an actual transmission rate for each stream of the one or more streams using the sub-total and the total; and

calculating a ratio between the desired transmission rate and the actual transmission rate.

21. A method comprising:

obtaining one or more streams;

processing the obtained one or more streams, wherein the processing includes;

dividing each of the one or more streams into a plurality of chunks, wherein each chunk of the plurality of chunks has an associated data payload and an associated chunk control byte including a chunk type identifier, and

writing to a chunk control byte rendering information relating to a media rendering state associated with a chunk and one or more subsequent chunks of the plurality of chunks, wherein a portion of the rendering information that remains unchanged from the chunk to the one or more subsequent chunks is written to the chunk control byte of the chunk but omitted from the chunk control byte of the one or more subsequent chunks in accordance with the chunk type identifiers of the chunk and the one or more subsequent chunks, such that the chunk control bytes have variable lengths; and

streaming the processed one or more streams from a media streaming server system.

22. The method of claim 21, wherein the writing comprises:

if the chunk is associated with a new stream from among the one or more streams, writing to the chunk control byte of the chunk

a new stream chunk type identifier as the chunk type identifier, and

rendering information relating to a media rendering state corresponding to the new stream chunk comprising a plurality of new parameters, wherein the media rendering state corresponding to the new stream chunk uses no parameters from a previous chunk of the one or more streams,

if the chunk is associated with a new media of the new stream, writing to the chunk control byte of the chunk

a new media chunk type identifier as the chunk type identifier, and

rendering information relating to a media rendering state corresponding to the new media chunk comprising a plurality of new parameters, wherein the media rendering state corresponding to the new media chunk uses at least one other parameter from a previous chunk of the new stream,

if the chunk is associated with a new time of the new stream, writing to the chunk control byte of the chunk

a new time chunk type identifier as the chunk type identifier, and

rendering information relating to a media rendering state corresponding to the new time chunk comprising at least one new parameter, wherein the media rendering state corresponding to the new time chunk uses a plurality of other parameters from a previous chunk of the new stream, and

else, writing to the chunk control byte a continuation chunk type identifier as the chunk type identifier, wherein a media rendering state corresponding to the continuation chunk uses rendering information from previous chunks,

wherein the parameters include one or more of a stream ID parameter, a length parameter, a media type parameter, and a time delta parameter.

23. The method of claim 22, wherein the streaming of the processed one or more streams comprises:

queuing the plurality of chunks for transmission based on a media type of each of the plurality of chunks, said queuing including staging the plurality of chunks according to a media type of the chunk, wherein the media type comprises one of an audio format chunk, a video format chunk, and a data format chunk; and

transmitting ones of the queued plurality of chunks based on a scheduling algorithm.

24. The method of claim 23, wherein the queuing comprises:

assigning a priority to each of the one or more streams;

totaling a priority for each stream;

calculating a desired transmission rate for each of the one or more streams using the assigned priority and the totaled priority;

counting a sub-total of the plurality of associated chunks transmitted per the one or more streams during a preset period;

counting a total number of the plurality of associated chunks transmitted during the preset period;

calculating an actual transmission rate for each stream of the one or more streams using the sub-total and the total; and

calculating a ratio between the desired transmission rate and the actual transmission rate.

25. The method of claim 21, further comprising storing the processed one or more streams to a storage system.

26. The method of claim 21, further comprising:

playing the processed one or more streams by using a media player of a computer system, said playing comprising, for each of the processed one or more streams,

reading, from a chunk control byte of a chunk from among the plurality of chunks of the processed stream, rendering information relating to the media rendering state associated with the chunk and one or more subsequent chunks; and

controlling a media rendering state for the media player based on the information read from the chunk control byte for playing the chunk and one or more subsequent chunks.

27. The method of claim 21, further comprising writing, to the associated chunk control byte of each of the plurality of chunks, an associated chunk identifier to indicate whether the associated data payload includes low-level protocol instructions or media content.