Real-time priority-based media communication

• US 7,272,658 B1
• Filed: 02/13/2003
• Issued: 09/18/2007
• Est. Priority Date: 02/13/2003
• Status: Active Grant

• Current Assignee: Adobe Inc.
• Original Assignee: Adobe Systems Incorporated (Adobe Inc.)
• Inventors: Edelman, Bradley, Lozben, Slavik, Shetty, Pritham, Gay, Jonathan
• Primary Examiner(s): Burgess; Glenton B.
• Assistant Examiner(s): Chea; Philip J.
• Application Number:
  US10/366,646
• Time in Patent Office: 1,678 Days
• Field of Search: 709/231, 725/103
• US Class Current: 709/231
• CPC Class Codes: H04L 47/10 Flow control; Congestion control
• CPC Class Codes: H04L 47/2416 Real-time traffic
• CPC Class Codes: H04L 47/2433 Allocation of priorities to traffic types
• CPC Class Codes: H04L 65/60 Network streaming of media packets
• CPC Class Codes: H04L 65/65 Network streaming protocols, e.g. real-time transport protocol [RTP] or real-time control protocol [RTCP]
• CPC Class Codes: H04L 65/762 at the source reformatting of additional data in video distribution servers H04N21/2355
• CPC Class Codes: H04N 21/2353 specifically adapted to content descriptors, e.g. coding, compressing or processing of metadata
• CPC Class Codes: H04N 21/2389 Multiplex stream processing, e.g. multiplex stream encrypting
• CPC Class Codes: H04N 21/262 Content or additional data distribution scheduling, e.g. sending additional data at off-peak times, updating software modules, calculating the carousel transmission frequency, delaying a video stream transmission, generating play-lists scheduling strategies for dispatcher in multiprogramming arrangements G06F9/4881; arrangements for scheduling broadcast services or broadcast-related services H04H60/06; flow control in packet networks H04L47/10; establishing a time schedule or organising the servicing of application requests H04L67/62
• CPC Class Codes: H04N 7/173 with two-way working, e.g. subscriber sending a programme selection signal

Abstract

Representative embodiments are disclosed of a real-time priority-based communication system and method for communicating media streams made up of multiple media message sub-streams, the communication system comprising a chunk configurator for dividing media message sub-streams into chunks, a state machine configured to translate between information regarding the media message sub-streams and the chunks and state associations to the information, the state associations written into a header for each of the chunks, a queue for holding the chunks waiting to be transmitted, and a processor for executing a scheduling algorithm, wherein the scheduling algorithm determines ones of the chunks in the queue to transmit next.

Claims

1. A real-time priority-based communication system for communicating media streams made up of multiple media message sub-streams, said communication system comprising:

a chunk configurator for dividing media message sub-streams into chunks;

a state machine configured to utilize information regarding said media message sub-streams and said chunks and to provide state associations corresponding to said information, said state associations written into a header for each of said chunks, said state associations classifying said chunks to identify possible purposes of each chunk, wherein the possible purposes identified by said state associations consist of;

a new stream chunk associated with a new media message sub-stream wherein said new stream chunk includes a plurality of new parameters in a corresponding one of said headers and uses no parameters from a previous header,

a new media chunk associated with a new media message sub-stream wherein said new media chunk includes a plurality of new parameters in a corresponding one of said headers and uses at least one other parameter from a previous header,

a new time chunk associated with said new media message sub-streams wherein said new time chunk includes at least one new parameter in a corresponding one of said headers and uses a plurality of other parameters from a previous header, and

a continuation chunk requiring no new parameters in a corresponding one of said headers and using a plurality of parameters from a previous header;

a queue for holding said chunks waiting to be transmitted; and

a processor for executing a scheduling algorithm, wherein said scheduling algorithm determines one of said chunks in said queue to transmit next.

2. The real-time priority-based communication system of claim 1 wherein said media message sub-streams are divided into said chunks having a preset maximum payload size.

3. The real-time priority-based communication system of claim 1 wherein said chunks comprise one or more of:

an audio format chunk;

a video format chunk; and

a data format chunk.

4. The real-time priority-based communication system of claim 1 wherein said state machine:

reads said information regarding said chunks and said media message sub-streams and translates said information into said state associations.

5. The real-time priority-based communication system of claim 4 wherein said parameters comprise one or more of:

a stream ID representing said label associated with said media message sub-streams;

a length of said media message sub-streams;

a media type indicator representing a type of media associated with said media message sub-streams; and

a time delta representing a time difference between successive of said chunks;

wherein a determination of which of said parameters are included in said state associations is determined by a value of said type indicator; and

wherein a header length is determined by a number of said parameters included with said state associations.

6. The real-time priority-based communication system of claim 1 wherein said plurality of new parameters included in a corresponding one of said headers of said new media includes new ones of one or more of:

a stream ID parameter;

a length parameter;

a media type indicator parameter; and

a time delta parameter; and

wherein said at least one new parameter included in a corresponding one of said headers of said new time chunk is a new time delta parameter.

7. The real-time priority-based communication system of claim 3 wherein said queue holds said chunks in sub-queues according to a format of said chunks.

8. The real-time priority-based communication system of claim 3 wherein said scheduling algorithm interleaves said audio format chunks, said video format chunks, and said data format chunks according to an efficiency algorithm.

9. The real-time priority-based communication system of claim 8 further comprising:

a priority assigned to each of;

said audio format chunks;

said video format chunks; and

said data format chunks;

a chunk counter for counting a sub-total of chunks associated with each of said media message sub-streams sent over a preset time period, said chunk counter also counting a total number of chunks sent over said preset time period.

10. The real-time priority-based communication system of claim 9 wherein said processor further:

totals a sub-stream priority for each of said media message sub-streams;

totals a stream priority for said media stream;

calculates a desired transmission percentage for each of said media message sub-streams using said sub-stream priority and said stream priority;

calculates an actual transmission percentage for each of said media message sub-streams using said sub-total of chunks and said total number of chunks; and

calculates a ratio between said desired transmission percentage and said actual transmission percentage.

11. The real-time priority-based communication system of claim 10 wherein said efficiency algorithm comprises:

a selection criteria for selecting said media message sub-stream having a lowest of said ratio;

a queue monitor for determining whether said chunk associated with said selected media message sub-stream is located in said queue; and

a queue manager for retrieving said selected chunk from said queue for transmission.

12. A method for communicating media streams, comprised of a plurality of sub-streams, in a real-time priority-based communication protocol, said method comprising:

dividing each one of said plurality of sub-streams into a plurality of chunks;

assigning states to each of said plurality of chunks representative of information regarding said associated sub-streams and said associated media streams, wherein said states indicate possible purposes of a corresponding chunk, and wherein the possible purposes indicated by said states consist of;

a new stream chunk associated with a new media message sub-stream wherein said new stream chunk includes a plurality of new parameters in a corresponding one of said headers and uses no parameters from a previous header,

a new media chunk associated with a new media message sub-stream wherein said new media chunk includes a plurality of new parameters in a corresponding one of said headers and uses at least one other parameter from a previous header,

a new time chunk associated with said new media message sub-streams wherein said new time chunk includes at least one new parameter in a corresponding one of said headers and uses a plurality of other parameters from a previous header, and

a continuation chunk requiring no new parameters in a corresponding one of said headers and using a plurality of parameters from a previous header;

writing said states into a header of each of said plurality of chunks;

queuing said plurality of chunks for transmission; and

transmitting ones of said plurality of chunks according to a scheduling algorithm.

13. The method of claim 12 further comprising:

receiving said transmitted plurality of chunks;

translating said states associated with each of said plurality of chunks into information regarding said associated sub-streams and said associated media streams; and

re-assembling said plurality of chunks into said associated plurality of sub-streams for replay of said media streams.

14. The method of claim 12 wherein said dividing includes:

maintaining a user-definable maximum payload size for each of said plurality of chunks.

15. The method of claim 12 wherein said assigning states includes:

assigning a unique chunk identification parameter for associating said chunk with its sub-stream, wherein said unique chunk identification parameter is smaller than a label associated with said sub-stream.

16. The method of claim 12 wherein said parameters include one or more of:

a stream ID parameter;

a length parameter;

a media type parameter; and

a time delta parameter.

17. The method of claim 12 wherein said queuing includes:

queuing said plurality of chunks according to a media type of said chunk, wherein said media type comprises one of;

an audio format chunk;

a video format chunk; and

a data format chunk.

18. The method of claim 17 wherein said scheduling algorithm further interleaves said audio format chunks, said video format chunks, and said data format chunks according to an interleaving algorithm.

19. The method of claim 12 further comprising:

assigning a priority to each sub-stream;

totaling a priority for each media stream;

calculating a desired transmission rate for each sub-stream using said assigned priority and said totaled priority;

counting a sub-total of chunks transmitted per said sub-stream during a preset period;

counting a total number of chunks transmitted during said preset period;

calculating an actual transmission rate for each sub-stream using said sub-total and said total; and

calculating a ratio between said desired transmission rate and said actual transmission rate.

20. The method of claim 19 wherein said scheduling algorithm includes:

determining a sub-stream having a lowest ratio;

checking said queue of said plurality of chunks for a chunk associated with said determined sub-stream; and

transmitting said chunk associated with said determined sub-stream when found in said queue.

21. A computer program product having a computer readable medium with computer program logic recorded thereon, said computer program product comprising:

code for breaking down each one of a plurality of media sub-streams making up a message stream into a plurality of associated chunks;

code for implementing a state machine that processes state assignments for each of said plurality of associated chunks representative of information regarding said media sub-streams and said message streams, wherein said state assignments indicate possible purposes of a corresponding chunk, and wherein the possible purposes indicated by said state assignments consist of;

a new stream chunk associated with a new media message sub-stream wherein said new stream chunk includes a plurality of new parameters in a corresponding one of said headers and uses no parameters from a previous header,

a new media chunk associated with a new media message sub-stream wherein said new media chunk includes a plurality of new parameters in a corresponding one of said headers and uses at least one other parameter from a previous header,

a new time chunk associated with said new media message sub-streams wherein said new time chunk includes at least one new parameter in a corresponding one of said headers and uses a plurality of other parameters from a previous header, and

a continuation chunk requiring no new parameters in a corresponding one of said headers and using a plurality of parameters from a previous header;

code for writing a corresponding one of said state assignments into a header of each of said plurality of associated chunks;

code for building a queue for said plurality of associated chunks waiting for transmission; and

code for transmitting ones of said plurality of associated chunks according to a scheduling algorithm.

22. The computer program product of claim 21 further comprising:

code for receiving said transmitted plurality of associated chunks;

code for translating said state assignments for each of said plurality of associated chunks into information regarding said media sub-streams and said message streams; and

code for re-assembling said plurality of associated chunks into said media sub-streams for replay of said message streams.

23. The computer program product of claim 21 wherein said code for breaking down includes:

code for setting a user-definable maximum payload size for each of said plurality of associated chunks.

24. The computer program product of claim 21 wherein said code for assigning states includes:

code for assigning a chunk ID for associating said chunk with its related media sub-stream, wherein said chunk ID is smaller than an ID related to said media sub-stream.

25. The computer program product of claim 21 wherein said parameters include at least one of:

a stream ID parameter;

a length parameter;

a media type parameter; and

a time delta parameter.

26. The computer program product of claim 21 wherein said queuing includes:

code for queuing said plurality of associated chunks according to a media type of said chunk, wherein said media type comprises one of;

an audio format chunk;

a video format chunk; and

a data format chunk.

27. The computer program product of claim 21 further comprising:

code for assigning a priority to each media sub-stream;

code for totaling a priority for each message stream;

code for calculating a desired transmission rate for each media sub-stream using said assigned priority and said totaled priority;

code for counting a sub-total of said plurality of associated chunks transmitted per said media sub-stream during a preset period;

code for counting a total number of said plurality of associated chunks transmitted during said preset period;

code for calculating an actual transmission rate for each media sub-stream using said sub-total and said total; and

code for calculating a ratio between said desired transmission rate and said actual transmission rate.

28. The computer program product of claim 27 wherein said scheduling algorithm includes:

code for selecting said media sub-stream having a lowest of said calculated ratio;

code for searching said queue for one of said plurality of associated chunks related to said selected media sub-stream; and

code for transmitting said one of said plurality of associated chunks related to said determined media sub-stream.