Virtual clustering and load balancing servers

• US 7,693,991 B2
• Filed: 01/16/2004
• Issued: 04/06/2010
• Est. Priority Date: 01/16/2004
• Status: Active Grant

• Current Assignee: X Corp. (f/k/a Twitter, Inc.) (X Holdings Corp.)
• Original Assignee: International Business Machines Corporation
• Inventors: Hartel, Dennis, Riina, Joseph F., Greenlee, Gordan G., Weingarten, Richard E.
• Primary Examiner(s): Srivastava; Vivek
• Assistant Examiner(s): MIRZA, ADNAN M
• Application Number:
  US10/758,538
• Publication Number:
  US 20050160133A1
• Time in Patent Office: 2,272 Days
• Field of Search: 709/223, 709/200, 709/224, 709/246
• US Class Current: 709/226
• CPC Class Codes: G06F 9/505 considering the load
• CPC Class Codes: G06F 9/5077 Logical partitioning of resources; Management or configuration of virtualized resources specific details on emulation or internal functioning of virtual machines G06F9/455
• CPC Class Codes: H04L 61/4523 using lightweight directory access protocol [LDAP]
• CPC Class Codes: H04L 67/1001 for accessing one among a plurality of replicated servers
• CPC Class Codes: H04L 67/1006 with static server selection, e.g. the same server being selected for a specific client
• CPC Class Codes: H04L 67/1008 based on parameters of servers, e.g. available memory or workload monitoring of computer activity G06F11/30
• CPC Class Codes: H04L 67/101 based on network conditions
• CPC Class Codes: H04L 67/1012 based on compliance of requirements or conditions with available server resources
• CPC Class Codes: H04L 67/1023 based on a hash applied to IP addresses or costs
• CPC Class Codes: H04L 67/1029 using data related to the state of servers by a load balancer

Abstract

A method and system is provided intelligent network dispatchers that are capable of routing requests to one or more servers (e.g., LDAP servers) based on the requested application or the identity of the requester. The servers may be grouped into virtual clusters so that a particular cluster may service requests associated with a particular application or based on a type of customer (e.g., preferred or general customer) in order to grade levels of service. Also, a system resource monitor may provide continual monitoring of servers in virtual clusters to determine when one or more servers are overburdened or under burdened based on statistical performance data. As a servers performance changes, the system resource monitor may advise the intelligent network dispatcher causing a change in the rate of requests to the overburdened/under burdened server. The server may be temporarily removed from the virtual cluster until performance levels normalize. Further, servers may be re-assigned to other virtual clusters on demand to provide increased capacity to a virtual cluster.

Claims

1. A method of processing queries in a network, comprising the steps of:

defining a plurality of virtual clusters from a plurality of servers;

routing a request to a virtual cluster of the plurality of virtual clusters based on predetermined criteria in order to allocate system resources;

removing at least one of the plurality of servers from the virtual cluster when at least one of the plurality of servers is burdened;

creating a new virtual cluster comprising only the removed at least one of the plurality of servers;

returning the removed at least one of the plurality of servers back to the virtual cluster when the at least one of the plurality of servers is unburdened;

monitoring performance of the plurality of servers; and

sending a report in response to workload at one of the plurality of servers exceeding a pre-determined threshold so that routing of further requests to the one of the plurality of servers is altered; and

removing the one of the plurality of servers from an associated virtual cluster and adding the one of the plurality of servers back into the associated virtual cluster in response to workload falling below the predetermined threshold.

2. The method of claim 1, wherein the sending a report sends a report to a network dispatcher and the network dispatcher performs the routing.

3. The method of claim 1, further comprising the steps of:

determining that one of the plurality of servers is overburdened based on statistics; and

reducing workload to the one of the plurality of servers if the statistics are above a threshold.

4. The method of claim 3, wherein the reducing step includes at least one of removing the one of a plurality of servers from one of the plurality of virtual clusters and limiting further requests from being routed to the one of a plurality of servers.

5. The method of claim 4, wherein the reducing step includes reassigning the one of a plurality of servers to another one of the plurality of virtual clusters.

6. The method of claim 1, wherein at least one of the plurality of servers is assigned to more than one of the plurality of virtual clusters.

7. The method of claim 1, wherein the predetermined criteria includes at least one of requester identity, requested application, time of day, day of week, and performance statistics.

8. The method of claim 7, wherein the requester identity is an internet address.

9. The method of claim 7, wherein the performance statistics include at least one of central processing unit (CPU) performance statistics, memory statistics, connection counts, throughput statistics, and response time statistics.

10. The method of claim 1, wherein the routing step includes selecting one of the plurality of virtual clusters for routing based on at least one of a requester'"'"'s identity and a requested application.

11. The method of claim 10, further including selecting one server from the one of the plurality of virtual clusters for routing based on statistics.

12. The method of claim 11, wherein the selecting is based on performance statistics.

13. The method of claim 1, wherein at least one of the plurality of servers is at least one of a lightweight directory access protocol (LDAP) server and a web application server.

14. The method of claim 1, wherein the routing uses rules based routing.

15. The method of claim 1, further comprising the steps of reassigning one of the plurality of servers from one of the plurality of virtual clusters to another one of the plurality of virtual clusters, wherein the one of the plurality of virtual clusters has a workload below a threshold and the another one of the plurality of virtual clusters has a workload above the predetermined threshold.

35. The method of claim 1, further comprising projecting a rate of routing to each of the plurality of servers.

36. The method of claim 35, further comprising adjusting the rate of routing based on a relative degree of overload on at least one of the plurality of servers.

37. The method of claim 36, further comprising providing early advisories when the rate of routing is projected to overload at least one of the plurality of servers.

38. The method of claim 1, further comprising determining when the plurality of servers in the virtual cluster are equivalently loaded over a predetermined workload threshold and more capacity is needed.

39. The method of claim 1, further comprising re-assigning at least one of the plurality of servers when the virtual cluster is above a predetermined cluster capacity rating.

16. A method for load balancing servers, comprising the steps of:

allocating a plurality of servers among a plurality of virtual clusters;

monitoring the plurality of virtual clusters for workload capacity;

removing at least one of the plurality of servers from the plurality of virtual clusters when at least one of the plurality of servers is burdened;

creating a new virtual cluster comprising only the removed at least one of the plurality of servers;

returning the removed at least one of the plurality of servers back to the plurality of virtual clusters when the at least one of the plurality of servers is unburdened; and

reassigning at least one server from one of the plurality of virtual clusters to another of the plurality of virtual clusters based on workload capacity of the at least one server in order to reallocate system resources,

wherein the monitoring step includes determining when a workload capacity of the one of the plurality of virtual clusters has crossed a threshold based on statistics associated with one of a plurality of virtual cluster'"'"'s performance; and

further comprising the step of identifying another of the plurality of virtual cluster having available workload capacity based on statistics associated with the virtual cluster'"'"'s performance and transferring at least one of the plurality of servers to the another of the virtual cluster.

17. The method of claim 16, wherein the reassigning at least one server includes one of:

removing the server entirely from the one of a plurality of virtual cluster, and

assigning the at least one server to both the one of a plurality of virtual clusters and the another of the plurality of virtual clusters.

18. The method of claim 16, further comprising routing a request to one of the plurality of virtual clusters based on one of the requestor'"'"'s identity, the requested application, and rules.

19. The method of claim 18, further comprising selecting one server assigned to the one of the plurality of virtual clusters based on statistics for routing the request.

20. A computer program product comprising a computer usable storage medium having readable program code embodied in the storage medium, the computer program product includes at least one component to:

define a plurality of virtual clusters from a plurality of servers;

route a request to a virtual cluster of the plurality of virtual clusters based on predetermined criteria to allocate system resources;

remove at least one of the plurality of servers from the virtual cluster when at least one of the plurality of servers is burdened;

create a new virtual cluster comprising only the removed at least one of the plurality of servers;

return the removed at least one of the plurality of servers back to the virtual cluster when the at least one of the plurality of servers is unburdened;

monitor performance of the plurality of servers;

send a report in response to workload at one of the plurality of servers exceeding a pre-determined threshold so that routing of further requests to the one of the plurality of servers is altered; and

remove the one of the pluar1ity of servers from an associated virtual cluster and add the one of the plurality of servers back into the associated virtual cluster in response to workload falling below the predetermined threshold.

21. The system of claim 20, wherein the at least one component sends a report to a network dispatcher and the network dispatcher performs the routing.

22. The system of claim 20, wherein the at least one component:

determines that one of the plurality of servers is overburdened based on statistics; and

reduces workload to the one of a plurality of servers if the statistics are above a threshold.

23. The system of claim 22, wherein the at least one component removes the one of a plurality of servers from one of the plurality of virtual clusters and limits further requests from being routed to the one of a plurality of servers.

24. The system of claim 23, wherein the at least one component reassigns the one of a plurality of servers to another one of the plurality of virtual clusters to reallocate the system resources.

25. The system of claim 20, wherein the at least one component assigns at least one of the plurality of servers to more than one of the plurality of virtual clusters.

26. The system of claim 20, wherein the predetermined criteria includes at least one of requester identity, requested application, time of day, day of week, performance statistics.

27. The system of claim 26, wherein the requester identity is a network address.

28. The system of claim 26, wherein the performance statistics include at least one of central processing unit (CPU) performance statistics, memory statistics, connection counts, throughput statistics, and response time statistics.

29. The system of claim 20, wherein the at least one component selects one of the plurality of virtual clusters for routing based on at least one of a requester'"'"'s identity, composite statistics, and a requested application.

30. The system of claim 20, wherein the at least one component selects a non over-burdened server from the one of the plurality of virtual clusters to process information.

31. The system of claim 30, wherein the at least one component selects based on performance statistics.

32. The system of claim 20, wherein at least one of the plurality of servers is one of a lightweight directory access protocol (LDAP) server and a web application server.

33. The system of claim 20, wherein the at least one component uses rules based routing.

34. The system of claim 20, wherein the at least one component reassigns one of the plurality of servers from one of the plurality of virtual clusters to another one of the plurality of virtual clusters, wherein the another of the plurality of virtual clusters has a workload below a threshold and the one of the plurality of virtual clusters has a workload above the predetermined threshold.