Method and apparatus for load sharing on a wide area network

• US 6,314,465 B1
• Filed: 03/11/1999
• Issued: 11/06/2001
• Est. Priority Date: 03/11/1999
• Status: Expired due to Term

• Current Assignee: WSOU Investments, LLC (WSOU Holdings, LLC)
• Original Assignee: Lucent Technologies, Inc. (Nokia Corporation)
• Inventors: Rangarajan, Sampath, Paul, Sanjoy
• Primary Examiner(s): Burgess, Glenton B.
• Assistant Examiner(s): Safavian, Seyed M.
• Application Number:
  US09/266,339
• Time in Patent Office: 971 Days
• Field of Search: 709/201, 709/202, 709/203, 709/218, 709/226, 709/227, 709/223, 709/229, 709/217, 709/219
• US Class Current: 709/226
• CPC Class Codes: H04L 67/1001 for accessing one among a plurality of replicated servers
• CPC Class Codes: H04L 67/1008 based on parameters of servers, e.g. available memory or workload monitoring of computer activity G06F11/30
• CPC Class Codes: H04L 67/101 based on network conditions
• CPC Class Codes: H04L 67/1012 based on compliance of requirements or conditions with available server resources
• CPC Class Codes: H04L 67/1034 Reaction to server failures by a load balancer

Abstract

Client'"'"'s (106-1-106-N, 107-1-107-M) on local area networks (102, 103) making requests to hot sites, which are connected on a wide area network (100) such as the Internet, are redirected through one of a possible plurality of different redirectors (101, 103) to one of a possible plurality of caching servers (S1, S2, S3), which each have responsibility for mapping one or more of the hot sites. Each request is probabilistically directed by one of the redirectors to one of the caching servers that map the requested hot site in accordance with weights that are determined for that redirector-hot site pair so as to minimize the average delay that all client requests across the network will encounter in making requests to all the cached hot sites. In order to determine the weights with which each redirector will redirect requests to the hot sites to the caching servers, statistics of access rates to each hot site are dynamically determined by each redirector in the network from the traffic flow and reported to a central management station (CMS) (115). Network delay is similarly measured by each redirector and reported to the CMS, and server delay is computed using a queuing model of each server. Using these parameters as inputs, a non-linear programming optimization problem is solved as a network flow problem in order to determine the weights for each redirector that will minimize the average delay. As the access rate statistics, as well as the network delay and server delay, dynamically change, the CMS, using the network flow algorithm, recalculates the weights and forwards them back to each redirector. In other embodiments, the redirector-logical item pair for which the redirector probabilistically directs client requests may be other than a hot site identity. For example, the logical items can be groups of clients or groups of documents, and the servers to which requests are forwarded can be web servers or caching servers.

Claims

1. A method of processing client requests through at least one redirector to a plurality of servers connected on a communications network to minimize an average delay associated with the client requests, at least some of the client requests being capable of being satisfied by more than one of the servers, the method comprising the steps of:

a) determining an access rate of requests associated with each of a plurality of redirector-logical item pairs;

b) determining a network delay between each of a plurality of clients and the plurality of servers;

c) determining a server delay incurred in processing a client request at each of the plurality of servers;

d) using the determined access rates of requests in step a), the network delays determined in step b) and the server delays determined in step c) as inputs, solving a non-linear program optimization problem to determine a set of weights associated with each of the plurality of redirector-logical item pairs so as to minimize the average delay associated with the client requests; and

e) probabilistically forwarding a client request through the at least one redirector to a server that can satisfy that request using the determined weights associated with the redirector-logical pair item.

2. The method of claim 1 wherein the step d) comprises the step of formulating and solving a minimum cost network flow problem.

3. The method of claim 2 further comprising the step of first determining an initial distribution that maps logical items onto the servers.

4. The method of claim 2 wherein the logical items are a plurality of hot sites.

5. The method of claim 4 wherein the servers are caching servers that each replicate at least one of the hot sites.

6. The method of claim 2 wherein the logical items are groups of clients.

7. The method of claim 6 wherein the groups of clients are determined by their IP addresses.

8. The method of claim 7 wherein the servers are web servers.

9. The method of claim 7 wherein the servers are caching servers.

10. The method of claim 2 wherein the logical items are groups of documents.

11. The method of claim 10 wherein the groups of documents are determined by their origin server IP addresses.

12. The method of claim 10 wherein the at least one redirector is a content-smart switch and the groups of documents are determined by their URLs.

13. The method of claim 11 wherein the servers are web servers.

14. The method of claim 11 wherein the servers are caching servers.

15. The method of claim 2 wherein the at least one redirector is a virtual server for a plurality of web servers.

16. The method of claim 2 wherein steps a) through d) are periodically repeated to determine a new set of weights associated with each redirector-logical item pair to thereafter be used by the at least one redirector.

17. The method of claim 2 wherein steps a) through d) are repeated when a change of an access rate of requests at a redirector is detected.

18. The method of claim 2 wherein steps a) through d) are repeated when a server failure is detected.

19. The method of claim 2 wherein steps a) through d) are repeated when a change in network delay is detected.

20. The method of claim 2 wherein steps a) through d) are repeated when a change in the delay at a server is detected.

21. The method of claim 2 wherein a central management station (CMS) connected on the communications network collects the determined access rates of requests in step a) and the network delay in step b) from the at least one redirector, and the server delay in step c) from the plurality of servers, and then performs step d), the method then further comprising the step of forwarding the determined weights to the at least one redirector.

22. In a system which processes client requests through at least one redirector to a plurality of servers connected on a communications network, at least some of the client requests being capable of being satisfied by more than one of the servers, apparatus for minimizing an average delay associated with the client requests, the apparatus comprising:

means for determining an access rate of requests associated with each of a plurality of redirector-logical item pairs;

means for determining a network delay between each of a plurality of clients and the plurality of servers;

means for determining a server delay incurred in processing a client request at each of the plurality of servers;

means for solving a non-linear programming optimization problem to determine a set of weights associated with each of the plurality of redirector-logical items pairs so as to minimize the average delay of client requests using the determined access rate of requests, the determined network delays, and the determined server delays as inputs to the problem; and

means for probabilistically forwarding a client request through the at least one redirector to a server in the system that can satisfy that request using the determined weights associated with the redirector-logical pair item.

23. The apparatus of claim 22 wherein the means for solving a non-linear optimization problem comprises means for formulating and solving a minimum cost network flow problem.

24. The apparatus of claim 23 further comprising means for determining an initial distribution that maps logical items onto the servers.

25. The apparatus of claim 23 wherein the logical items are a plurality of hot sites.

26. The apparatus of claim 25 wherein the servers are caching servers that each replicate at least one of the hot sites.

27. The apparatus of claim 23 wherein the logical items are groups of clients.

28. The apparatus of claim 27 wherein the groups of clients are determined by their IP addresses.

29. The apparatus of claim 28 wherein the servers are web servers.

30. The apparatus of claim 28 wherein the servers are caching servers.

31. The apparatus of claim 23 wherein the logical items are groups of documents.

32. The apparatus of claim 31 wherein the groups of documents are determined by their origin server IP addresses.

33. The apparatus of claim 31 wherein the at least one redirector is a content-smart switch and the groups of documents are determined by their URLs.

34. The apparatus of claim 32 wherein the servers are web servers.

35. The apparatus of claim 32 wherein the servers are caching servers.

36. The apparatus of claim 23 wherein the at least one redirector is a virtual server for a plurality of web servers.

37. The apparatus of claim 23 wherein the means for solving the non-linear optimization problem periodically determines a new set of weights associated with each redirector-logical item pair to thereafter be used by the at least one redirector.

38. The apparatus of claim 23 wherein the means for solving the non-linear optimization problem determines a new set of weights when a change of an access rate at a redirector is detected.

39. The apparatus of claim 23 wherein the means for solving the non-linear optimization problem determines a new set of weights when a server failure is detected.

40. The apparatus of claim 23 wherein the means for solving the non-linear optimization problem determines a new set of weights when a change in network delay is detected.

41. The apparatus of claim 23 wherein the means for solving the non-linear optimization problem determines a new set of weights when a change in the delay at a server is detected.

42. The apparatus of claim 23 further comprising means for forwarding the determined weights to the at least one redirector.

43. In a system which processes client requests through at least one redirector to a plurality of servers connected on a communications network, at least some of the client requests being capable of being satisfied by more than one of the servers, a method of determining a set of weights with which the at least one redirector will probabilistically forward client requests to the server in the system that can satisfy the requests comprising the steps of:

a) determining an access rate of requests associated with each of a plurality of redirector-logical item pairs;

b) determining a network delay between each of a plurality of clients and the plurality of servers;

c) determining a server delay incurred in processing a client request at each of the plurality of servers; and

d) using the determined access rates of requests in step a), the network delays determined in step b) and the server delays determined in step c) as inputs, solving a non-linear program optimization problem to determine the set of weights associated with each of the plurality of redirector-logical item pairs so as to minimize the average delay associated with the client requests.

44. The method of claim 43 wherein the step d) comprises the step of formulating and solving a minimum cost network flow problem.

45. The method of claim 44 further comprising the step of first determining an initial distribution that maps logical items onto the servers.

46. The method of claim 44 wherein the logical items are a plurality of hot sites.

47. The method of claim 46 wherein the servers are caching servers that each replicate at least one of the hot sites.

48. The method of claim 44 wherein the logical items are groups of clients.

49. The method of claim 48 wherein the groups of clients are determined by their IP addresses.

50. The method of claim 49 wherein the servers are web servers.

51. The method of claim 49 wherein the servers are caching servers.

52. The method of claim 44 wherein the logical items are groups of documents.

53. The method of claim 52 wherein the groups of documents are determined by their origin server IP addresses.

54. The method of claim 52 wherein the at least one redirector is a content-smart switch and the groups of documents are determined by their URLs.

55. The method of claim 53 wherein the servers are web servers.

56. The method of claim 53 wherein the servers are caching servers.

57. The method of claim 44 wherein the at least one redirector is a virtual server for a plurality of web servers.

58. The method of claim 44 wherein steps a) through d) are periodically repeated to determine a new set of weights associated with each redirector-logical item pair to thereafter be used by the at least one redirector.

59. The method of claim 44 wherein steps a) through d) are repeated when a change of an access rate of requests at a redirector is detected.

60. The method of claim 44 wherein steps a) through d) are repeated when a server failure is detected.

61. The method of claim 44 wherein steps a) through d) are repeated when a change in network delay is detected.

62. The method of claim 44 wherein steps a) through d) are repeated when a change in the delay at a server is detected.

63. The method of claim 44 wherein a central management station (CMS) connected on the communications network collects the determined access rates of requests in step a) and the network delay in step b) from the at least one redirector, and the server delay in step c) from the plurality of servers, and then performs step d), the method then further comprising the step of forwarding the determined weights to the at least one redirector.

64. A system for processing client requests to a plurality of servers connected on a communications network, at least some of the client requests being capable of being satisfied by more than one of the servers, the system comprising:

a least one redirector; and

at least one processor performing the steps of;

a) determining an access rate of requests associated with each of a plurality of redirector-logical item pairs;

b) determining a network delay between each of a plurality of clients and the plurality of servers;

c) determining a server delay incurred in processing a client request at each of the plurality of servers; and

d) using the determined access rates of requests in step a), the network delays determined in step b) and the server delays determined in step c) as inputs, solving a non-linear program optimization problem to determine a set of weights associated with each of the plurality of redirector-logical item pairs so as to minimize the average delay associated with the client requests;

the at least one redirector using the determined weights associated with each redirector-logical item pair to probabilistically forward each client request to a server that can satisfy that request.

65. The system of claim 64 wherein the at least one processor performs step d) by formulating and solving a minimum cost network flow problem.

66. The system of claim 65 wherein the at least one processor first performs a step of determining an initial distribution that maps logical items onto the servers.

67. The system of claim 65 wherein the logical items are a plurality of hot sites.

68. The system of claim 67 wherein the servers are caching servers that each replicate at least one of the hot sites.

69. The system of claim 65 wherein the logical items are groups of clients.

70. The system of claim 69 wherein the groups of clients are determined by their IP addresses.

71. The system of claim 70 wherein the servers are web servers.

72. The system of claim 70 wherein the servers are caching servers.

73. The system of claim 65 wherein the logical items are groups of documents.

74. The system of claim 73 wherein the groups of documents are determined by their origin server IP addresses.

75. The apparatus of claim 73 wherein the at least one redirector is a content-smart switch and the groups of documents are determined by their URLs.

76. The system of claim 74 wherein the servers are web servers.

77. The system of claim 74 wherein the servers are caching servers.

78. The system of claim 65 wherein the at least one redirector is a virtual server for a plurality of web servers.

79. The system of claim 65 wherein the at least one processor performs steps a) through d) periodically to determine a new set of weights associated with each redirector-logical item pair to be thereafter used by the at least one redirector.

80. The system of claim 65 wherein the at least one processor repeats steps a) through d) when a change of an access rate of requests at a redirector is detected.

81. The system of claim 65 wherein the at least one processor repeats steps a) through d) when a server failure is detected.

82. The system of claim 65 wherein the at least one processor repeats steps a) thought d) when a change in network delay is detected.

83. The system of claim 65 wherein the at least one processor repeats steps a) through d) when a change in the delay at a server is detected.