Highly scalable system and method of regulating internet traffic to server farm to support (min,max) bandwidth usage-based service level agreements
First Claim
1. A system for controlling and managing Internet server farm traffic through a plurality of servers, the server farm traffic arriving at a server farm as inbound traffic organized by customer (i) and traffic type (j) and leaving the server farm as outbound traffic, the system being operable to control and manage the outbound traffic in accordance with outbound bandwidth usage-based service level agreements of form (Bmin,Bmax) by computing a target rate (Rt) for each customer traffic type (i,j) that supports the outbound bandwidth usage-based service level agreements of form (Bmin,Bmax) and then admitting a portion of the inbound traffic at an admitted rate (Ra) while rejecting at a rejected rate (Rr) a remaining portion of the inbound traffic that if processed would cause the outbound traffic for the customer traffic tape (i,j) to exceed the target rate (Rt), the system comprising:
- means for collecting the admitted rate (Ra) of inbound traffic for each customer traffic type (i,j);
means for collecting the rejected rate (Rr) of inbound traffic for each customer traffic type (i,j);
means for collecting the outbound traffic (B) for each customer traffic type (i,j);
means for computing an expected bandwidth usage (b) per TCP connection request for each customer traffic type (i,j);
means for using the expected bandwidth usage (b) to compute the target rate (Rt) for each customer traffic type (i,j) that supports the outbound bandwidth usage-based service level agreements of form (Bmin,Bmax);
limiter means for admitting inbound traffic based on the target rate (Rt), for rejecting inbound traffic that exceeds the target rate (Rt), and for tracking the volume of admitted inbound traffic (Ra) and the volume of rejected inbound traffic (Rr) for each customer traffic type (i,j);
means for relaying the target rates (Rt) for inbound traffic to the limiter means; and
means for dispatching the admitted inbound traffic (Ra) to the servers.
1 Assignment
0 Petitions
Accused Products
Abstract
A highly scalable system and method for supporting (mim,max) based Service Level Agreements (SLA) on outbound bandwidth usage for a plurality of customers whose applications (e.g.,Web sites) are hosted by a server farm that consists of a very large number of servers. The system employs a feedback system that enforces the outbound link bandwidth SLAs by regulating the inbound traffic to a server or server farm. Inbound traffic is admitted to servers using a rate denoted as Rt(i,j), which is the amount of the ith customer'"'"'s jth type of traffic that can be admitted within a service cycle time to servers which support the ith customer. A centralized device computes Rt(i,j) based on the history of admitted inbound traffic to servers, the history of generated outbound traffic from servers, and the SLAs of various customers. The Rt(i,j) value is then relayed to one or more inbound traffic limiters that regulate the inbound traffic using the rates Rt(i,j) in a given service cycle time. The process of computing and deploying Rt(i,j) values is repeated periodically. In this manner, the system provides a method by which differentiated services can be provided to various types of traffic, the generation of output from a server or a server farm is avoided if that output cannot be delivered to end users, and revenue can be maximized when allocating bandwidth beyond the minimums.
58 Citations
33 Claims
-
1. A system for controlling and managing Internet server farm traffic through a plurality of servers, the server farm traffic arriving at a server farm as inbound traffic organized by customer (i) and traffic type (j) and leaving the server farm as outbound traffic, the system being operable to control and manage the outbound traffic in accordance with outbound bandwidth usage-based service level agreements of form (Bmin,Bmax) by computing a target rate (Rt) for each customer traffic type (i,j) that supports the outbound bandwidth usage-based service level agreements of form (Bmin,Bmax) and then admitting a portion of the inbound traffic at an admitted rate (Ra) while rejecting at a rejected rate (Rr) a remaining portion of the inbound traffic that if processed would cause the outbound traffic for the customer traffic tape (i,j) to exceed the target rate (Rt), the system comprising:
-
means for collecting the admitted rate (Ra) of inbound traffic for each customer traffic type (i,j);
means for collecting the rejected rate (Rr) of inbound traffic for each customer traffic type (i,j);
means for collecting the outbound traffic (B) for each customer traffic type (i,j);
means for computing an expected bandwidth usage (b) per TCP connection request for each customer traffic type (i,j);
means for using the expected bandwidth usage (b) to compute the target rate (Rt) for each customer traffic type (i,j) that supports the outbound bandwidth usage-based service level agreements of form (Bmin,Bmax);
limiter means for admitting inbound traffic based on the target rate (Rt), for rejecting inbound traffic that exceeds the target rate (Rt), and for tracking the volume of admitted inbound traffic (Ra) and the volume of rejected inbound traffic (Rr) for each customer traffic type (i,j);
means for relaying the target rates (Rt) for inbound traffic to the limiter means; and
means for dispatching the admitted inbound traffic (Ra) to the servers. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15)
-
-
16. A system for controlling and managing Internet server farm traffic through a plurality of servers, the server farm traffic arriving at a server farm as inbound traffic organized by customer (I) and traffic type (j) and leaving the server farm as outbound traffic, the system being operable to control and manage the outbound traffic in accordance with outbound bandwidth usage-based service level agreements of form (Bmin,Bmax) by computing a target rate (Rt) for each customer traffic type (i,j) that supports the outbound bandwidth usage-based service level agreements of form (Bmin,Bmax) and then admitting a portion of the inbound traffic at an admitted rate (Ra) while rejecting at a rejected rate (Rr) a remaining portion of the inbound traffic that if processed would cause the outbound traffic for the customer traffic type (i,j) to exceed the target rate (Rt), the system being further operable to control and manage the outbound traffic in accordance with a server resource manager that establishes an absolute bound (Rbound) of a target rate (Rt) for each customer traffic type (i,j), the system comprising:
-
an inbound traffic scheduler device operable to collect the admitted rate (Ra) and the rejected rata (Rr) of inbound traffic for each customer traffic type (i,j), collect the bound (Rbound) from any server resource manager and collect the outbound traffic (B) for each customer traffic type (i,j), the inbound traffic scheduler device being further operable to compute an expected bandwidth usage (b) per request for each customer traffic type (i,j) and use the expected bandwidth usage (b) to compute a target rate (Rt) for inbound traffic for each customer traffic type (i,j) to support the outbound bandwidth usage-based service level agreements of form (Bmin,Bmax);
an inbound traffic limiter operable to receive the target rate (Rt) from the inbound traffic scheduler device, admit inbound traffic based on the target rate (Rt), track the volume of admitted inbound traffic (Ra) and the volume of rejected inbound traffic (Rr) for each customer traffic type (i,j), and relay the amount of admitted inbound traffic (Ra) and the amount of rejected traffic (Rr) to the inbound traffic scheduler device; and
an inbound traffic dispatching network operable to classify incoming traffic, the inbound traffic dispatching network being controlled by the inbound traffic limiter to selectively dropping packets arriving in the inbound traffic limiter if the inbound traffic exceeds the target rate (Rt), the inbound traffic dispatching network further being comprised of a high-speed LAN with dispatchers to dispatch the admitted inbound traffic (Ra) to the servers. - View Dependent Claims (17, 18, 19, 20, 21, 22, 23, 24)
-
-
25. A method for controlling and managing Internet server farm traffic through a plurality of servers, the server farm traffic arriving at a server farm as inbound traffic organized by customer (I) and traffic type (j) and leaving the server farm as outbound traffic that is controlled and managed in accordance with outbound bandwidth usage-based service level agreements (Bmin(i,j),Bmax(i,j)) by computing a target rate (Rt) for each customer traffic type (i,j) that supports the outbound bandwidth usage-based service level agreements of form (Bmin(i,j),Bmax(i,j) and then admitting a portion of the inbound traffic at an admitted rate (Ra) while rejecting at a rejected rate (Rr) a remaining portion of the inbound traffic that if processed would cause the outbound traffic for the customer traffic type (i,j) to exceed the target rate (Rt), the method comprising the steps of:
-
collecting the admitted rate (Ra(i,j)) of inbound traffic for each customer traffic type (i,j);
collecting the rejected rate (Rr(i,j)) of inbound traffic for each customer traffic type (i,j);
collecting the outbound traffic (B(i,j)) for each customer traffic type (i,j);
collecting the absolute bound (Rbound(i,j)) on the target rate (Rt(i,j)) for each customer traffic type (i,j);
computing an expected bandwidth usage (b(i,j)) per TCP connection request for each customer traffic type (i,j);
using the expected bandwidth usage (b(i,j)) to compute the target rate (Rt(i,j)) for each customer traffic type (i,j) based on the admitted rate (Ra(i,j)), the rejected rate (Rr(i,j)), the outbound traffic (B(i,j)), the expected bandwidth usage (b(i,j)) and the outbound bandwidth usage-based service level agreements (Bmin(i,j),Bmax(i,j));
admitting inbound traffic based on the target rate (Rt(i,j)), rejecting inbound traffic that exceeds the target rate (Rt(i,j)) and tracking the volume of admitted inbound traffic (Ra(i,j)) and the volume of rejected inbound traffic (Rr(i,j)) for each customer traffic type (i,j);
relaying the target rates (Rt(i,j)) for inbound traffic to the limiter means; and
dispatching the admitted inbound traffic (Rp(i,j)) to the servers. - View Dependent Claims (26, 27, 28, 29, 30, 31, 32, 33)
-
Specification