Real-Time Priority-Based Media Communication

• US 20090327510A1
• Filed: 09/04/2009
• Published: 12/31/2009
• Est. Priority Date: 02/13/2003
• Status: Active Grant

• Current Assignee: Adobe Inc.
• Original Assignee: Adobe Systems Incorporated (Adobe Inc.)
• Inventors: Lozben, Slavik, Edelman, Bradley A., Shetty, Pritham, Gay, Jonathan
• Granted Patent: US 8,065,426 B2
• Time in Patent Office: Days
• US Class Current: 709/231
• CPC Class Codes: H04L 47/10 Flow control; Congestion control
• CPC Class Codes: H04L 47/2416 Real-time traffic
• CPC Class Codes: H04L 47/2433 Allocation of priorities to traffic types
• CPC Class Codes: H04L 65/60 Network streaming of media packets
• CPC Class Codes: H04L 65/65 Network streaming protocols, e.g. real-time transport protocol [RTP] or real-time control protocol [RTCP]
• CPC Class Codes: H04L 65/762 at the source reformatting of additional data in video distribution servers H04N21/2355
• CPC Class Codes: H04N 21/2353 specifically adapted to content descriptors, e.g. coding, compressing or processing of metadata
• CPC Class Codes: H04N 21/2389 Multiplex stream processing, e.g. multiplex stream encrypting
• CPC Class Codes: H04N 21/262 Content or additional data distribution scheduling, e.g. sending additional data at off-peak times, updating software modules, calculating the carousel transmission frequency, delaying a video stream transmission, generating play-lists scheduling strategies for dispatcher in multiprogramming arrangements G06F9/4881; arrangements for scheduling broadcast services or broadcast-related services H04H60/06; flow control in packet networks H04L47/10; establishing a time schedule or organising the servicing of application requests H04L67/62
• CPC Class Codes: H04N 7/173 with two-way working, e.g. subscriber sending a programme selection signal

Abstract

Representative embodiments are disclosed of a real-time priority-based communication system and method for communicating media streams made up of multiple media message sub-streams, the communication system comprising a chunk configurator for dividing media message sub-streams into chunks, a state machine configured to translate between information regarding the media message sub-streams and the chunks and state associations to the information, the state associations written into a header for each of the chunks, a queue for holding the chunks waiting to be transmitted, and a processor for executing a scheduling algorithm, wherein the scheduling algorithm determines ones of the chunks in the queue to transmit next.

Claims

1-31. -31. (canceled)

32. A computer readable medium storing a program to cause at least one processor to perform operations comprising:

obtaining one or more streams;

processing the obtained one or more streams, wherein the processing includes;

dividing each of the one or more streams into a plurality of chunks, wherein each chunk has an associated data payload and an associated chunk control byte including a chunk type identifier, and

writing to a chunk control byte rendering information relating to a media rendering state associated with the chunk and one or more subsequent chunks, wherein a portion of the rendering information that remains unchanged from one chunk to a subsequent chunk is omitted from the rendering information written in the chunk control byte of the subsequent chunk, such that the chunk control bytes have variable lengths; and

outputting the processed one or more streams.

33. The computer readable medium of claim 32, wherein the writing comprises:

if the chunk is associated with a new stream from among the one or more streams, inserting to the chunk control byte

a new stream chunk type identifier, and

rendering information relating to a media rendering state corresponding to the new stream chunk comprising a plurality of new parameters, wherein the media rendering state corresponding to the new stream chunk uses no parameters from a previous chunk of the one or more streams;

if the chunk is associated with a new media of the new stream, inserting to the chunk control byte

a new media chunk type identifier, and

rendering information relating to a media rendering state corresponding to the new media chunk comprising a plurality of new parameters, wherein the media rendering state corresponding to the new media chunk uses at least one other parameter from a previous chunk of the new stream;

if the chunk is associated with a new time of the new stream, inserting to the chunk control byte

a new time chunk type identifier, and

rendering information relating to a media rendering state corresponding to the new time chunk comprising at least one new parameter, wherein the media rendering state corresponding to the new time chunk uses a plurality of other parameters from a previous chunk of the new stream; and

else, inserting to the chunk control byte a continuation chunk type identifier, wherein a media rendering state corresponding to the continuation chunk uses rendering information from previous chunks.

34. The computer readable medium of claim 33, wherein the parameters include one or more of a stream ID parameter, a length parameter, a media type parameter, and a time delta parameter.

35. The computer readable medium of claim 32, the operations further comprising:

writing, to the associated chunk control byte of each of the plurality of chunks, an associated chunk identifier to indicate whether the associated data payload includes low-level protocol instructions or media content.

36. The computer readable medium of claim 34, wherein the outputting comprises:

storing the plurality of chunks of the processed one or more streams to a storage device.

37. The computer readable medium of claim 34, wherein the outputting comprises:

playing the plurality of chunks of the processed one or more streams by a media player of a computing device.

38. The computer readable medium of claim 37, wherein the playing comprises:

reading, from a chunk control byte of a chunk from among the plurality of chunks, rendering information relating to the media rendering state associated with the chunk and one or more subsequent chunks; and

controlling a media rendering state for the media player based on the information read from the chunk control byte for playing the chunk and one or more subsequent chunks.

39. The computer readable medium of claim 34, wherein the outputting comprises:

streaming the plurality of chunks of the processed one or more streams from a media streaming server computer.

40. The computer readable medium of claim 39, the operations further comprising:

queuing the plurality of chunks for transmission based on a media type of each of the plurality of chunks; and

transmitting ones of the queued plurality of chunks based on a scheduling algorithm.

41. The computer readable medium of claim 40, wherein the queuing comprises:

staging the plurality of chunks according to a media type of the chunk, wherein the media type comprises one of an audio format chunk, a video format chunk, and a data format chunk.

42. The computer readable medium of claim 41, the operations further comprising:

assigning a priority to each of the one or more streams;

totaling a priority for each stream;

calculating a desired transmission rate for each of the one or more streams using the assigned priority and the totaled priority;

counting a sub-total of the plurality of associated chunks transmitted per the one or more streams during a preset period;

counting a total number of the plurality of associated chunks transmitted during the preset period;

calculating an actual transmission rate for each stream of the one or more streams using the sub-total and the total; and

calculating a ratio between the desired transmission rate and the actual transmission rate.

43. A system comprising:

a user device;

a storage device; and

one or more computer systems operable to interact with the user device and with the storage device, the one or more computer systems are configured to perform operations including;

obtaining one or more streams;

processing the obtained one or more streams, wherein the processing includes;

dividing each of the one or more streams into a plurality of chunks, wherein each chunk has an associated data payload and an associated chunk control byte including a chunk type identifier, and

writing to a chunk control byte rendering information relating to a media rendering state associated with the chunk and one or more subsequent chunks, wherein a portion of the rendering information that remains unchanged from one chunk to a subsequent chunk is omitted from the rendering information written in the chunk control byte of the subsequent chunk, such that the chunk control bytes have variable lengths; and

outputting the processed one or more streams.

44. The system of claim 43, wherein the writing operation further comprises operations including:

if the chunk is associated with a new stream from among the one or more streams, inserting to the chunk control byte

a new stream chunk type identifier, and

rendering information relating to a media rendering state corresponding to the new stream chunk comprising a plurality of new parameters, wherein the media rendering state corresponding to the new stream chunk uses no parameters from a previous chunk of the one or more streams;

if the chunk is associated with a new media of the new stream, inserting to the chunk control byte

a new media chunk type identifier, and

rendering information relating to a media rendering state corresponding to the new media chunk comprising a plurality of new parameters, wherein the media rendering state corresponding to the new media chunk uses at least one other parameter from a previous chunk of the new stream;

if the chunk is associated with a new time of the new stream, inserting to the chunk control byte

a new time chunk type identifier, and

rendering information relating to a media rendering state corresponding to the new time chunk comprising at least one new parameter, wherein the media rendering state corresponding to the new time chunk uses a plurality of other parameters from a previous chunk of the new stream; and

else, inserting to the chunk control byte a continuation chunk type identifier, wherein a media rendering state corresponding to the continuation chunk uses rendering information from previous chunks.

45. The system of claim 44, wherein the parameters include one or more of a stream ID parameter, a length parameter, a media type parameter, and a time delta parameter.

46. The system of claim 45, wherein the one or more computer systems consist of one computer, the user device is a user interface device, and the one computer comprises the user interface device, wherein the one computer is further configured to perform operations including:

storing the plurality of chunks of the processed one or more streams to the storage device.

47. The system of claim 45, wherein the one or more computer systems consist of one computer, the user device is a user interface device, and the one computer comprises the user interface device, wherein the one computer is further configured to perform operations including:

playing the plurality of chunks of the processed one or more streams by a media player of the user interface device, wherein the playing comprises;

reading, from a chunk control byte of a chunk from among the plurality of chunks, rendering information relating to the media rendering state associated with the chunk and one or more subsequent chunks; and

controlling a media rendering state for the media player based on the information read from the chunk control byte for playing the chunk and one or more subsequent chunks.

48. The system of claim 45, wherein:

the one or more computer systems comprise a media streaming server operable to interact with the user device through a data communication network,

the user device is operable to interact with the server as a client, and the user device comprises a personal computer running a media player,

wherein, the media streaming server is configured to perform operations including;

streaming the plurality of chunks of the processed one or more streams from the media streaming server to the personal computer.

49. The system of claim 48, wherein the media streaming server is further configured to perform operations including:

queuing the plurality of chunks for transmission based on a media type of each of the plurality of chunks; and

transmitting ones of the queued plurality of chunks based on a scheduling algorithm.

50. The system of claim 49, wherein the media streaming server is further configured to perform operations including:

staging the plurality of chunks according to a media type of the chunk, wherein the media type comprises one of an audio format chunk, a video format chunk, and a data format chunk.

51. The system of claim 50, wherein the media streaming server is further configured to perform operations including:

assigning a priority to each of the one or more streams;

totaling a priority for each stream;

calculating a desired transmission rate for each of the one or more streams using the assigned priority and the totaled priority;

counting a sub-total of the plurality of associated chunks transmitted per the one or more streams during a preset period;

counting a total number of the plurality of associated chunks transmitted during the preset period;

calculating an actual transmission rate for each stream of the one or more streams using the sub-total and the total; and

calculating a ratio between the desired transmission rate and the actual transmission rate.