Resource Allocation in Multi-Stream IP Network for Optimized Quality of Service

• US 20060156201A1
• Filed: 03/23/2006
• Published: 07/13/2006
• Est. Priority Date: 05/31/2000
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Zhang, Ya-Qin Jr., Zhang, Qian, Zhu, Wenwu
• Granted Patent: US 7,559,078 B2
• Time in Patent Office: Days
• US Class Current: 714/776
• CPC Class Codes: H04L 47/15 in relation to multipoint traffic arrangements for broadcast or multicast in data networks H04L12/18
• CPC Class Codes: H04L 47/193 at the transport layer, e.g. TCP related
• CPC Class Codes: H04L 47/2416 Real-time traffic
• CPC Class Codes: H04L 47/263 Rate modification at the source after receiving feedback
• CPC Class Codes: H04L 47/28 in relation to timing considerations
• CPC Class Codes: H04L 47/70 Admission control; Resource allocation
• CPC Class Codes: H04L 47/762 triggered by the network
• CPC Class Codes: H04L 47/801 Real time traffic
• CPC Class Codes: H04L 47/822 Collecting or measuring resource availability data
• CPC Class Codes: H04N 21/234318 by decomposing into objects, e.g. MPEG-4 objects
• CPC Class Codes: H04N 21/234354 by altering signal-to-noise ratio parameters, e.g. requantization
• CPC Class Codes: H04N 21/2381 Adapting the multiplex stream to a specific network, e.g. an Internet Protocol [IP] network transmission of MPEG streams over ATM H04L12/5601
• CPC Class Codes: H04N 21/2402 Monitoring of the downstream path of the transmission network, e.g. bandwidth available traffic monitoring in data switching networks H04L43/00; monitoring data switching networks utilization H04L43/0876
• CPC Class Codes: H04N 21/2662 Controlling the complexity of the video stream, e.g. by scaling the resolution or bitrate of the video stream based on the client capabilities
• CPC Class Codes: H04N 21/4381 Recovering the multiplex stream from a specific network, e.g. recovering MPEG packets from ATM cells transmission of MPEG streams over ATM H04L12/5601
• CPC Class Codes: H04N 21/44209 Monitoring of downstream path of the transmission network originating from a server, e.g. bandwidth variations of a wireless network arrangements for maintenance or administration in data switching networks involving bandwidth and capacity management H04L41/0896
• CPC Class Codes: H04N 21/64322 IP
• CPC Class Codes: H04N 21/6582 Data stored in the client, e.g. viewing habits, hardware capabilities, credit card number arrangements where receivers interact with the broadcast H04H20/38
• CPC Class Codes: H04N 7/173 with two-way working, e.g. subscriber sending a programme selection signal

Abstract

A resource allocation of multiple compressed AV streams delivered over the Internet is disclosed that achieves end-to-end optimal quality through a multimedia streaming TCP-friendly transport (MSTFP) protocol that adaptively estimates the network bandwidth while smoothing the sending rate. Resources allocated dynamically according to a media encoding distortion and network degradation algorithm. A scheme is also disclosed for dynamically estimating the available network bandwidth for streaming of objects, such as MPEG4 multiple video objects, in conjunction with the MSTFP protocol. The scheme can account for packet-loss rates to minimize end-to-end distortion for media delivery.

Claims

1. A computer system for transmitting a mixed media data stream in packets, including audio and video objects, between a sender and a receiver through a connection over a network, the computer system adapted to implement a method comprising:

monitoring, at the receiver, transmission characteristics of the connection between the server and the receiver;

estimating available bandwidth at the sender based upon the transmission characteristics of the connection monitored at the receiver;

allocating a global buffer for the mixed media data stream to be transmitted from the sender to the receiver as a function of the estimated available bandwidth at the sender;

pre-encoding a portion of each Video Object Plane (VOP) in the global buffer with respect to a quantization parameter (QP) of the VOP;

encoding the VOP in the global buffer based on the QP;

updating a rate distortion model based upon the QP and packet loss rate;

performing a frame skipping function after the VOP encoding; and

transmitting from the sender to the receiver the encoded video object plane in the global buffer at a regulated sender transmission rate from the sender as a function of the estimated available bandwidth at the sender;

wherein pre-encoding a portion of each VOP with respect to the QP of the VOP further comprises adjusting the QP of the VOP; and

wherein the QP of the VOP is adjusted with respect to a texture parameter, r, as the number of bits which will be used to encode the VOP wherein;

$r=\frac{p_1\times <br><br>\mathrm{MAD}}{\mathrm{QP}}+\frac{p_2\times <br><br>\mathrm{MAD}}{{\mathrm{QP}}^2};<br><br>$

p₁ and p₂ are control parameters; and

MAD is a mean absolute distortion in which a total target bit rate for all objects in the global buffer are allocated proportionally to motion, size, and square of MAD.

2. The computer system as defined in claim 1, wherein the method implemented by the computer system further comprises:

receiving the encoded video object plane at the receiver from the connection;

demultiplexing the encoded video object plane into coded video and audio streams;

inputting the coded video and audio streams, respectively, into video and audio decoders;

inputting the decoded video and audio streams to a media mixer; and

inputting the mixed video and audio streams output from the media mixer to an output device.

3. The computer system as defined in claim 1, wherein the adjusting of the QP of the MOP is performed by changing the QP to a values in a range from 1 to 31 depending upon the estimated available bandwidth at the receiver.

4. The computer system as defined in claim 1, wherein the computer system comprises at least one of (i) one or more servers or (ii) one or more clients.

5. A computer system for transmitting a mixed media data stream in packets, including audio and video objects, between a sender and a receiver through a connection over a network, the computer system adapted to implement a method comprising:

monitoring, at the receiver, transmission characteristics of the connection between the server and the receiver;

estimating available bandwidth at the sender based upon the transmission characteristics of the connection monitored at the receiver;

allocating a global buffer for the mixed media data stream to be transmitted from the sender to the receiver as a function of the estimated available bandwidth at the sender;

pre-encoding a portion of each Video Object Plane (VOP) in the global buffer with respect to a quantization parameter (QP) of the VOP;

encoding the VOP in the global buffer based on the QP;

updating a rate distortion model based upon the QP and packet loss rate;

performing a frame skipping function after the VOP encoding; and

transmitting from the sender to the receiver the encoded video object plane in the global buffer at a regulated sender transmission rate from the sender as a function of the estimated available bandwidth at the sender; and

wherein;

the sender sends data to the receiver in through a connection over a packet switched network in a sender packet having a sender header that includes;

a packet sequence number;

a timestamp indicating the time when the sender packet was sent (ST1); and

the size of the sender packet (PacketSize);

the receiver sends data to the sender through the connection over the packet switched network in a receiver packet having a receiver header that includes;

the time interval that the sender packet spent in the receiver side (Δ

RT);

the timestamp of the sender packet sent from the sender (ST1);

an estimate, calculated by the receiver, of a packet-loss rate; and

the rate at which data is received at the receiver;

monitoring transmission characteristics of the connection between server and receiver comprises;

estimating a round trip time of the sender packet from the sender to the receiver (RTT) based on ST1 and Δ

RT;

estimating a time out interval (TO) before which the sender should retransmit to the receiver a sender packet of data that has not been received by the receiver;

estimating a probability that a packet of data will be lost (P_L);

estimating the present available network bandwidth at which the receiver can receive data from the sender (rvcrate) as a function of the PacketSize, the RTT, the P_L, and the TO;

deriving the present sending rate of data from the sender to the receiver ({overscore (currate)});

setting an updated sending rate of data from the sender to the receiver (currate), wherein;

if revrate is greater than {overscore (currate)}, then deriving currate as a function {overscore (currate)}, PacketSize, and RTT; and

if rcvrate is not greater than {overscore (currate)}, then setting currate to be less than rcvrate; and

wherein;

RTT=α

×

{overscore (RTT)}+(1−

α

)×

(now−

ST1−

Δ

RT); and

now is the timestamp indicating the time at which the receiver packet was received in the sender; and

a is a weighting parameter.

6. The computer system as defined in claim 5, wherein the computer system comprises at least one of (i) one or more servers or (ii) one or more clients.

7. A computer system for transmitting a mixed media data stream in packets, including audio and multiple video objects (MVOs), between a sender and a receiver through a connection over a network, the computer system adapted to implement a method comprising:

monitoring transmission characteristics of one or more encoded video object planes through the connection between the sender and the receiver;

estimating, from the transmission characteristics, an available bandwidth (RT) at the sender;

allocating, as a function of the RT, a portion of the mixed media data stream to a global buffer, encoding a video object plane from the global buffer based upon a rate distortion function that accounts for packet loss rate between sender and receiver;

.

updating the rate distortion function based upon results of the encoded video object plane and upon a memory containing results of one or more previously encoded video object planes;

after the encoding the MVOs in the video object plane, performing a frame skipping function; and

transmitting, at the estimated available bandwidth, the encoded video object plane from the sender to the receiver;

wherein allocating a portion of the mixed media data stream to a global buffer comprises;

W_cur=max(((W_prev+B_prev)×

R_T/R_old−

R_T/F),0), as the global buffer size R_old/2, is changed to R_T/2, wherein;

B_prev is the number of bits spent in the previous time instant B_prev,

R_old/2 is the previous size of the global buffer;

W_prev is the previous occupancy of the global buffer; and

F is the video frame rate.

8. The computer system as defined in claim 7, wherein allocating a portion of the mixed media data stream to a global buffer comprises the allocation of an output target rate from the global buffer among each of video and audio data streams so as to yield the target bits for an individual object in the data stream.

9. The computer system as defined in claim 7, wherein the method implemented by the computer system farther comprises:

receiving the encoded video object plane at the receiver from the connection;

demultiplexing the encoded video object plane into coded video and audio streams;

inputting the coded video and audio streams, respectively, into video and audio decoders;

inputting the decoded video and audio streams to a media mixer; and

inputting the mixed video and audio streams output from the media mixer to an output device.

10. The computer system as defined in claim 7, wherein the computer system comprises at least one of (i) one or more servers or (ii) one or more clients.