On-demand global server load balancing system and method of use

• US 9,160,792 B2
• Filed: 04/05/2005
• Issued: 10/13/2015
• Est. Priority Date: 04/05/2005
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Bali, Bahri B., Greenlee, Gordan G., Weingarten, Richard E.
• Primary Examiner(s): Gillis, Brian J
• Assistant Examiner(s): MOREAU, AUSTIN J
• Application Number:
  US11/098,854
• Publication Number:
  US 20060224725A1
• Time in Patent Office: 3,843 Days
• Field of Search: 709/223, 709/224, 709/226, 709/232
• US Class Current: 1/1
• CPC Class Codes: H04L 61/4511 using domain name system [DNS]
• CPC Class Codes: H04L 67/1001 for accessing one among a plurality of replicated servers
• CPC Class Codes: H04L 67/1008 based on parameters of servers, e.g. available memory or workload monitoring of computer activity G06F11/30
• CPC Class Codes: H04L 67/1021 based on client or server locations
• CPC Class Codes: H04L 67/1029 using data related to the state of servers by a load balancer

Abstract

A method and system for measuring performance and balancing workload of sites, virtual clusters and servers within the virtual clusters is provided. A monitor measures and computes performance metrics at each level. Servers are individually measured for performance and based on average response time of servers that are members of a virtual cluster, a virtual cluster performance metric may also be computed. Further, the performance of a site may be profiled based on the virtual cluster metrics and/or server performance metrics. Based at least in part on these performance metrics, a domain oriented request may be resolved and routed based in part on the performance metrics at each level virtual cluster and server. In this manner, a request may be directed to a destination server to achieve higher performance and efficiencies for the requesting client while at the same time balancing workload among possible servers available to service the request.

Claims

1. A method of processing queries in a network, comprising determining, by a computing device, virtual cluster membership information of a plurality of servers in the network, the virtual cluster membership information identifying each of the plurality of servers as a member in a respective one of a plurality of virtual server clusters, wherein the virtual server clusters are dynamically determined, logical groupings of the plurality of servers;

measuring, by the computing device, respective performance information of the plurality of servers;

computing, by the computing device, respective performance metrics corresponding to the plurality of servers;

computing, by the computing device, respective virtual cluster performance metrics corresponding to the plurality of virtual clusters based on the respective virtual cluster membership information and the respective performance metrics of the plurality of servers belonging to the plurality of virtual clusters, wherein the respective virtual cluster performance metrics include average response times over a predetermined period of time of the plurality of servers identified as members in each of the plurality of virtual server clusters;

resetting the virtual cluster performance metrics as definitions of respective virtual clusters in the virtual cluster membership information are dynamically changed;

routing a request to one of the plurality of servers based on the computed performance metric to balance a workload among the one or more servers;

routing the request to one of a plurality of site selectors associated with one of a plurality of sites, which includes assuring that a percentage of traffic routed overall to the selected server within a virtual cluster is proportionate to a weighted performance of the individual servers; and

determining, by the computing device, performance metrics of the one of the plurality of sites by requesting performance metrics from site selectors.

2. The method of claim 1, further comprising the step of storing the performance metric on the one or more servers.

3. The method of claim 1, wherein the performance metric is a response time metric and stored on the one or more servers.

4. The method of claim 1, wherein the routing a request step comprises basing the routing on a distance factor.

5. The method of claim 1, further comprising the steps of:

routing the request to a site selector for selecting a site from a plurality of sites;

selecting the site for processing the request based on site performance metrics; and

routing the request to an address associated with the site for processing.

6. The method of claim 5, wherein the address is an address of a network dispatcher.

7. The method of claim 1, wherein the routing step comprises:

selecting one virtual cluster from the one or more virtual clusters based on the virtual cluster performance metric; and

routing the request to the one of the one or more servers that is a member of the selected one virtual cluster.

8. The method of claim 7, wherein the virtual cluster performance metric is an indication of performance of the selected one virtual cluster based on an average of response times of member servers of the selected one virtual cluster.

9. The method of claim 1, further comprising:

monitoring one or more sites for performance;

computing site performance metrics for the one or more sites; and

storing the site performance metrics on at least one site selector for basing the routing.

10. The method of claim 1, wherein the measuring performance step comprises sending a random request to each of the one or more servers and measuring the amount of time to reply.

11. A method of routing requests in a network, comprising:

computing, by a computer device, respective performance metrics for one of a plurality of sites, a first virtual server cluster of a plurality of virtual server clusters, and a first server of a plurality of servers, wherein the virtual server clusters are dynamically determined, logical groupings of the plurality of servers;

obtaining, by the computer device, virtual cluster membership information identifying the first server as a member of the first virtual server cluster;

routing, by the computer device, a request to the site, the first virtual cluster and the first server based on the virtual cluster membership information and the respective performance metrics for balancing workload among the plurality of servers, wherein the respective performance metrics include a response time metric associated with each of the site, the virtual cluster, and the server of the plurality of servers, wherein the routing the request includes assuring that a percentage of traffic routed overall to the selected server within a virtual cluster is proportionate to a weighted performance of the individual servers;

routing the request to one of a plurality of site selectors associated with the one of the plurality of sites; and

determining, by the computing device, performance metrics of the one of the plurality of sites by requesting performance metrics from at least one of the plurality of site selectors.

12. A system for routing requests in a network, comprising:

a computing platform comprised of hardware and one or more computer readable memory storage devices;

computer readable program code stored on the one or more computer readable memory storage devices for execution by the computing platform, the program code comprising;

program code that measures performance of one or more servers in a plurality of virtual clusters;

program code that generates a server performance metric corresponding to each of the one or more servers;

program code that determines virtual cluster membership information corresponding of the one or more servers, the virtual cluster membership information identifying the one or more servers as a member in a respective one of the plurality of virtual server clusters, wherein the virtual server clusters are dynamically determined, logical groupings of the plurality of servers;

program code that computes a virtual cluster performance metric corresponding to each one of the plurality of virtual clusters based on the virtual cluster membership information and the server performance metric for each of the one or more servers belonging to the plurality of virtual clusters, wherein the virtual cluster performance metric includes an average of response times over a predetermined period of time of the one or more servers identified as members in each of the plurality of virtual server clusters;

program code that routes a request to one of the one or more servers based on the virtual cluster performance metric and the server performance metric to balance a workload among the one or more servers;

program code that routes the request to one of a plurality of site selectors associated with one of a plurality of sites; and

program code that determines site performance metrics of the one of the plurality of sites by requesting performance metrics from at least one of the site selectors,

wherein the virtual cluster performance metrics are reset as the definitions of respective virtual clusters in the virtual cluster membership information are dynamically changed and the routing the request includes assuring that a percentage of traffic routed overall to the selected server within a virtual cluster is proportionate to a weighted performance of the individual servers.

13. The system of claim 12, further comprising program code that stores the server performance metric on the one or more servers.

14. The system of claim 12, wherein the server performance metric is at least any one of a response time metric, a processing metric, a memory metric, a connection count metric, which is stored on the one or more servers.

15. The system of claim 12, wherein the program code that routes performs the routing based on a distance factor.

16. The system of claim 12, further comprising:

program code that routes the request to a site selector for selecting a site from a plurality of sites;

program code that selects the site for processing the request based on site performance metrics; and

program code that routes the request to an address associated with the site for processing.

17. A computer program product comprising a computer readable storage device having computer readable program code stored on the computer readable storage device, the computer program code comprising:

program code that computes respective virtual cluster performance metrics for each of a plurality of sites, a first virtual cluster of a plurality of virtual clusters, and a first server of a plurality of servers, wherein the virtual server clusters are dynamically determined, logical groupings of the plurality of servers;

program code that obtains virtual cluster membership information identifying the first server as a member of the first virtual server cluster;

program code that routes a request to one of the plurality of sites, the first virtual cluster and the first server based on the virtual cluster membership information and the respective performance metrics for balancing workload among the plurality of servers;

program code that routes the request to one of a plurality of site selectors associated with one of a plurality of sites; and

program code that determines site performance metrics of the one of the plurality of sites by requesting performance metrics from at least one of the plurality of site selectors,

wherein the virtual cluster performance metrics are reset as the definitions of respective virtual clusters in the virtual cluster membership information are dynamically changed, the routing the request includes assuring that a percentage of traffic routed overall to the selected server within a virtual cluster is proportionate to a weighted performance of the individual servers, and the respective site performance metrics include a response time metric associated with each of the site, the virtual cluster, and the server of the plurality of servers.

18. A system for balancing workload, comprising at least one hardware component configured to:

collect server performance data corresponding to a plurality of servers associated with a site, the server performance data including respective utilization statistics of the plurality of servers;

obtain virtual cluster membership information corresponding to the plurality of servers, the virtual cluster membership information identifying each of the plurality of servers as a member of a respective one of a plurality of virtual server clusters associated with the site, wherein the virtual server clusters are dynamically determined, logical groupings of the plurality of servers;

generate, based on the utilization statistics, respective server performance metrics corresponding to the plurality of servers, wherein the virtual server clusters are dynamically determined, logical groupings of the plurality of servers;

generate, based on the virtual cluster membership information and the server performance metrics, respective virtual cluster performance metric corresponding to the plurality of virtual server clusters associated with the site, wherein the virtual cluster performance metrics are reset as the definitions of respective virtual clusters in the virtual cluster membership information are dynamically changed;

select one of the plurality of virtual server clusters to handle a client request based on the generated virtual cluster performance metrics;

select a first server of the plurality of servers that is a member of the selected virtual server cluster to handle the client request based on the generated server performance metrics;

route the client request to the selected first server;

route the request to one of a plurality of site selectors associated with one of a plurality of sites, wherein the routing the request includes assuring that a percentage of traffic routed overall to the selected server within a virtual cluster is proportionate to a weighted performance of the individual servers; and

determine site performance metrics of the one of the plurality of sites by requesting site performance metrics from at least one of the plurality of site selectors.

19. The system of claim 18, wherein:

the collecting the server performance data comprises sending a plurality of searches to each one of the plurality of servers and measuring a plurality of response times of each one of the plurality of servers;

the generating the server performance metric comprises determining an average response time of each one of the plurality of servers; and

the generating the virtual cluster performance metric comprises determining an average response time of each one of the plurality of virtual server clusters.

20. A method comprising:

receiving, by a computing device, a client request;

routing the request to one of a plurality of site selectors associated with one of a plurality of sites;

determining, by the one of the plurality of site selectors, site performance metrics of the one of the plurality of sites by requesting performance metrics from other site selectors;

selecting, by the one of the plurality of site selectors, a site to handle the client request from the one of the plurality of sites based on the site performance metrics;

routing, by the one of the plurality of site selectors, the client request to a network dispatcher at the selected site, wherein the selected site comprises a plurality of virtual server clusters and each one of the plurality of virtual server clusters comprises a plurality of servers, wherein the virtual server clusters are dynamically determined, logical groupings of the plurality of servers;

determining, by the network dispatcher, server performance metrics of the plurality of servers;

obtaining, by the network dispatcher, virtual cluster membership information corresponding to the plurality of servers, the virtual cluster membership information identifying each of the plurality of servers as a member of a respective one of the plurality of virtual server clusters;

determining, by the network dispatcher, virtual cluster performance metrics of the plurality of virtual server clusters based on the virtual cluster membership information and the server performance metrics of the plurality of servers;

wherein the virtual cluster performance metrics are reset as the definitions of respective virtual clusters in the virtual cluster membership information are dynamically changed;

after the selecting the selected site, selecting, by the network dispatcher, one of the plurality of virtual server clusters for handling the request based on the virtual cluster performance metrics;

after the selecting the selected virtual server cluster, selecting, by the network dispatcher, one of the plurality of servers of the selected virtual server cluster for handling the request based on the server performance metrics; and

routing, by the network dispatcher, the request to the selected server;

wherein the routing the request includes assuring that a percentage of traffic routed overall to the selected server within a virtual cluster is proportionate to a weighted performance of the individual servers.

21. The method of claim 20, wherein:

the site performance metrics are based on average site response time and a distance weighting factor;

the virtual cluster performance metrics are based on average cluster response time; and

the server performance metrics are based on average server response time and at least one of;

a processing metric, a memory metric, and a connection count metric.