Resource allocation in multi-stream IP network for optimized quality of service

• US 7,574,726 B2
• Filed: 03/23/2006
• Issued: 08/11/2009
• Est. Priority Date: 05/31/2000
• Status: Expired due to Fees

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Zhang, Qian, Zhang, Ya-Qin, Zhu, Wenwu
• Primary Examiner(s): Kelley; Christopher
• Assistant Examiner(s): Brown; Reuben M
• Application Number:
  US11/277,260
• Publication Number:
  US 20060156200A1
• Time in Patent Office: 1,237 Days
• Field of Search: 370/249, 370/252, 370/519, 375/240.22, 375/240.23, 375/240.29, 375/240.03, 714746-748, 725/94, 725/95, 725/126
• US Class Current: 725/95
• CPC Class Codes: H04L 47/15 in relation to multipoint traffic arrangements for broadcast or multicast in data networks H04L12/18
• CPC Class Codes: H04L 47/193 at the transport layer, e.g. TCP related
• CPC Class Codes: H04L 47/2416 Real-time traffic
• CPC Class Codes: H04L 47/263 Rate modification at the source after receiving feedback
• CPC Class Codes: H04L 47/28 in relation to timing considerations
• CPC Class Codes: H04L 47/70 Admission control; Resource allocation
• CPC Class Codes: H04L 47/762 triggered by the network
• CPC Class Codes: H04L 47/801 Real time traffic
• CPC Class Codes: H04L 47/822 Collecting or measuring resource availability data
• CPC Class Codes: H04N 21/234318 by decomposing into objects, e.g. MPEG-4 objects
• CPC Class Codes: H04N 21/234354 by altering signal-to-noise ratio parameters, e.g. requantization
• CPC Class Codes: H04N 21/2381 Adapting the multiplex stream to a specific network, e.g. an Internet Protocol [IP] network transmission of MPEG streams over ATM H04L12/5601
• CPC Class Codes: H04N 21/2402 Monitoring of the downstream path of the transmission network, e.g. bandwidth available traffic monitoring in data switching networks H04L43/00; monitoring data switching networks utilization H04L43/0876
• CPC Class Codes: H04N 21/2662 Controlling the complexity of the video stream, e.g. by scaling the resolution or bitrate of the video stream based on the client capabilities
• CPC Class Codes: H04N 21/4381 Recovering the multiplex stream from a specific network, e.g. recovering MPEG packets from ATM cells transmission of MPEG streams over ATM H04L12/5601
• CPC Class Codes: H04N 21/44209 Monitoring of downstream path of the transmission network originating from a server, e.g. bandwidth variations of a wireless network arrangements for maintenance or administration in data switching networks involving bandwidth and capacity management H04L41/0896
• CPC Class Codes: H04N 21/64322 IP
• CPC Class Codes: H04N 21/6582 Data stored in the client, e.g. viewing habits, hardware capabilities, credit card number arrangements where receivers interact with the broadcast H04H20/38
• CPC Class Codes: H04N 7/173 with two-way working, e.g. subscriber sending a programme selection signal

Abstract

A resource allocation of multiple compressed AV streams delivered over the Internet is disclosed that achieves end-to-end optimal quality through a multimedia streaming TCP-friendly transport (MSTFP) protocol that adaptively estimates the network bandwidth while smoothing the sending rate. Resources allocated dynamically according to a media encoding distortion and network degradation algorithm. A scheme is also disclosed for dynamically estimating the available network bandwidth for streaming of objects, such as MPEG4 multiple video objects, in conjunction with the MSTFP protocol. The scheme can account for packet-loss rates to minimize end-to-end distortion for media delivery.

Claims

1. A method for transmitting a mixed media data stream in packets, including audio and video objects, between a sender and a receiver through a connection over a network, the method comprising:

monitoring transmission characteristics of the connection between the sender and the receiver, wherein monitoring transmission characteristics comprises;

estimating a round trip time of a sender packet from the sender to the receiver (RTT) based on a timestamp indicating the time when the sender packet was sent (ST1) and a time interval that the sender packet spent in the receiver side (Δ

RT);

estimating a time out interval (TO) before which the sender should retransmit to the receiver a sender packet of data that has not been received by the receiver;

estimating a probability that a packet of data will be lost (P_L);

estimating the present available network bandwidth at which the receiver can receive data from the sender (rvcrate);

deriving the present sending rate of data from the sender to the receiver ( currate); and

setting an updated sending rate of data from the sender to the receiver (currate), wherein;

if rcvrate is greater than currate, then deriving currate as a function currate, PacketSize, and RTT; and

if rcvrate is not greater than currate, then setting currate to be less than rcvrate;

allocating a global buffer for the mixed media data stream to be transmitted from the sender to the receiver as a function of the estimated available bandwidth at the sender;

pre-encoding a portion of each Video Object Plane (VOP) in the global buffer with respect to a quantization parameter (QP) of the VOP;

encoding the VOP in the global buffer based on the QP;

updating a rate distortion model based upon the QP and packet loss rate;

performing a frame skipping function after the VOP encoding;

transmitting the encoded VOP to the receiver in a sender packet over a packet switched network at a regulated sender transmission rate as a function of the estimated available bandwidth at the sender, wherein a header of the sender packet includes;

(1) a packet sequence number, (2) the timestamp indicating the time when the sender packet was sent (ST1), and (3) the size of the sender packet (PacketSize); and

receiving a receiver packet from the receiver over the packet switched network, wherein a header of the receiver packet includes;

(1) the time interval that the sender packet spent in the receiver side (Δ

RT), (2) the timestamp of the sender packet sent from the sender (ST1), (3) an estimate, calculated by the receiver, of a packet-loss rate, and (4) the rate at which data is received at the receiver.

2. The method as defined in claim 1, further comprising:

receiving the encoded video object plane at the receiver from the connection;

demultiplexing the encoded video object plane into coded video and audio streams;

inputting the coded video and audio streams, respectively, into video and audio decoders;

inputting the decoded video and audio streams to a media mixer; and

inputting the mixed video and audio streams output from the media mixer to an output device.

3. The method as defined in claim 1, wherein pre-encoding a portion of each VOP with respect to the QP of the VOP further comprises adjusting the QP of the VOP.

4. The method as defined in claim 1, wherein:

5. The method as defined in claim 4, wherein:

the P_L is further smoothed by a filter that weights the n most recent measured packet loss rates by;

6. A computer-readable media comprising computer-executable instructions for performing the method as recited in claim 1.

7. One or more computer-readable media having computer-readable instructions stored thereon, wherein the computer-readable media includes:

a first set of elements to describe a server in communication through a connection of a packet switched network to a client;

a second set of elements to describe the transmission of a mixed media data stream, including audio and multiple video objects (MVOs), from the server to the client through a connection over a packet switched network; and

a code segment that, when executed;

monitors transmission characteristics of the connection between server and receiver, wherein monitoring transmission characteristics comprises;

estimating a round trip time of a sender packet from the sender to the receiver (RTT);

estimating a time out interval (TO) before which the sender should retransmit to the receiver a sender packet of data;

estimating a probability that a packet of data will be lost (P_L);

estimating the present available network bandwidth at which the receiver can receive data from the sender (rvcrate);

deriving the present sending rate of data from the sender to the receiver ( currate); and

setting an updated sending rate of data from the sender to the receiver (currate), wherein;

if rcvrate is greater than currate, then deriving currate as a function currate, and RTT; and

if rcvrate is not greater than currate, then setting currate to be less than rcvrate

allocates a global buffer for the mixed media data stream to be transmitted stream from the sender to the receiver as a function of the estimated available bandwidth at the sender;

pre-encodes a portion of each Video Object Plane (VOP) in the global buffer with respect to a quantization parameter (QP) of the VOP;

encodes the VOP in the global buffer based on the QP;

updates a rate distortion model based upon the QP and packet loss rate;

performs a frame skipping function after the VOP encoding;

transmits the encoded VOP to the receiver in a sender packet, wherein a header of the sender packet includes;

(1) a packet sequence number, (2) the timestamp indicating the time when the sender packet was sent (ST1), and (3) the size of the sender packet (PacketSize); and

receives a receiver packet from the receiver, wherein a header of the receiver packet includes;

(1) the time interval that the sender packet spent in the receiver side (Δ

RT), (2) the timestamp of the sender packet sent from the sender (ST1), (3) an estimate, calculated by the receiver, of a packet-loss rate, and (4) the rate at which data is received at the receiver.