System, method, and architecture for dynamic server power management and dynamic workload management for multi-server environment

• US 20060265608A1
• Filed: 02/28/2006
• Published: 11/23/2006
• Est. Priority Date: 09/27/2000
• Status: Abandoned Application

• Current Assignee: Huron IP LLC
• Original Assignee: Huron IP LLC
• Inventors: Fung, Henry T.
• Application Number:
  US11/364,820
• Publication Number:
  US 20060265608A1
• Time in Patent Office: Days
• US Class Current: 713/300
• CPC Class Codes: G06F 1/3203 Power management, i.e. event-based initiation of a power-saving mode
• CPC Class Codes: G06F 1/3209 Monitoring remote activity, e.g. over telephone lines or network connections
• CPC Class Codes: G06F 1/3221 of disk drive devices
• CPC Class Codes: G06F 3/0625 Power saving in storage systems
• CPC Class Codes: G06F 3/0634 by changing the state or mode of one or more devices
• CPC Class Codes: G06F 3/0689 Disk arrays, e.g. RAID, JBOD
• CPC Class Codes: Y02D 10/00 Energy efficient computing, e.g. low power processors, power management or thermal management

Abstract

Network architecture, computer system and/or server, circuit, device, apparatus, method, and computer program and control mechanism for managing power consumption and workload in computer system and data and information servers. Further provides power and energy consumption and workload management and control systems and architectures for high-density and modular multi-server computer systems that maintain performance while conserving energy and method for power management and workload management. Dynamic server power management and optional dynamic workload management for multi-server environments is provided by aspects of the invention. Modular network devices and integrated server system, including modular servers, management units, switches and switching fabrics, modular power supplies and modular fans and a special backplane architecture are provided as well as dynamically reconfigurable multi-purpose modules and servers. Backplane architecture, structure, and method that has no active components and separate power supply lines and protection to provide high reliability in server environment.

Claims

1. A computer system comprising:

a plurality of server computers each having at least one processor and an activity monitor identifying a level of activity indicator for said at least one processor;

each of said server computers being operable in;

(i) a first mode having a first maximum performance level and a first power consumption rate, (ii) a second mode having a second maximum performance level lower than said first maximum performance level and a second power consumption rate lower than said first power consumption rate, and (iii) a third mode having a third maximum performance level lower than said second maximum performance level and a third power consumption rate lower than said second power consumption rate; and

a power manager;

(i) coupled to each of said server computers and receiving said level of activity information from each of said plurality of computers;

(ii) analyzing said plurality of received level of activity information;

(iii) determining an operating mode for each of said server computers selected from said first mode, second mode, and third mode based on said analyzed activity information and predetermined policies; and

(iv) generating commands to each of said plurality of server computers directing each of said plurality of server computers to operate in said determined operating mode.

2. A computer system comprising:

a plurality of computers each having at least one processor and an activity monitor identifying a level of activity indicator for said at least one processor;

each of said computers being operable in;

(i) a first mode having a first maximum performance level and a first power consumption rate, and (ii) a third mode having a third maximum performance level lower than said first maximum performance level and a third power consumption rate lower than said first power consumption rate; and

a power manager;

(i) coupled to each of said computers and receiving said level of activity information from each of said plurality of computers;

(ii) analyzing said plurality of received level of activity information;

(iii) determining an operating mode for each of said computers selected from said first mode and third mode based on said analyzed activity information and predetermined policies; and

(iv) generating commands to each of said plurality of computers directing each of said plurality of computers to operate in said determined operating mode.

3. The computer system in claim 2, wherein:

each of said computers further being operable in (iii) a second mode having a second maximum performance level intermediate between said first maximum performance level and said third maximum performance level and a second power consumption rate intermediate between said first power consumption rate and said third power consumption rate; and

said power manager further determining an operating mode for each of said computers selected from said first mode, said second mode, and said third mode based on said analyzed activity information and said predetermined policies.

4. The computer system in any of claims 2 or 3, wherein:

said computers comprise servers.

5. The computer system in any of claims 2, 3, or 4, further comprising a power manager computer providing said power manager.

6. The computer system in any of claims 2, 3, or 4 wherein a selected one of said plurality of computers designated as a master providing said power manager.

7. The computer system in any of claims 2 or 3, wherein said activity monitor comprises an activity monitor that monitors an activity selected from the set of activities consisting of:

a program application layer activity, a network layer activity, a physical layer activity, and combinations thereof.

8. A system as in claim 7, wherein at the physical level the number of processor idle threads executed within a predetermined period of time are measured to determine processor loading and the processor performance is adjusted to by altering the operating mode to substantially match the level of processor loading.

9. The computer system in claim 2, wherein said activity monitor comprises a network layer activity monitoring TCP/IP protocol data packets;

and processor performance is incrementally lowered by said power manager using said mode control until data packets start dropping indicating that the processor performance is at the limit of adequacy and then increasing the processor performance by a specified increment to act as a safety margin to provide reliable communication of the packets.

10. The computer system in claim 7, wherein said application layer activity monitor comprises monitoring use of a port address within said computers, said monitoring including counting or measuring a number of times a specific port address is being requested within a predetermined period of time, and in response to that counting or measurement, placing a sufficient amount of computer performance to meet the performance requirement for each application requesting the port address.

11. The computer system in claim 7, wherein said application layer activity monitor comprises monitoring use of a port address within said computers.

12. The computer system in claim 7, wherein said network layer activity monitor comprises monitoring use of a TCP/IP protocol within said computers.

13. The computer system in claim 7, wherein said physical layer activity monitor comprises monitoring the execution of idle threads within said computers.

14. The computer system in claim 7, wherein said physical layer activity monitor comprises monitoring counting activities having particular activity values within said computers.

15. The computer system in claim 3, wherein:

said first mode operation is characterized by operating said processor at a first processor clock frequency and a first processor core voltage, said second mode operation is characterized by operating said processor at a second processor clock frequency and a second processor core voltage, and said third mode operation is characterized by operating said processor at a third processor clock frequency and a third processor core voltage;

said second mode of operation being further characterized in that said second processor clock frequency and said second processor core voltage in combination consuming less power than said first processor clock frequency and said first processor core voltage in combination, and said third processor clock frequency and said third processor core voltage in combination consuming less power than said second processor clock frequency and said second processor core voltage in combination.

16. A system as in claim 15, wherein performance of a group of said computers configured as physical network devices forming a single logical device are power managed by reducing the performance and power consumption of each constituent physical device in predetermined equal increments or predetermined unequal increments.

17. A system as in claim 15, wherein network device loading and quality of service (QoS) are measured for a plurality of physical network devices organized as a single logical network device.

18. The computer system in claim 15, wherein said third processor clock frequency is less than said second processor clock frequency which is less than said first processor clock frequency.

19. The computer system in claim 18, wherein said second processor core voltage is less than said first processor core voltage.

20. The computer system in claim 19, wherein said third processor core voltage is less than said second processor core voltage.

21. The computer system in claim 15, wherein said third processor clock frequency is less than said second processor clock frequency which is less than said first processor clock frequency;

and said second processor core voltage is less than said first processor core voltage.

22. The computer system in claim 2, wherein:

each of said computers further being operable in (iii) a second mode having a second maximum performance level intermediate between said first maximum performance level and said third maximum performance level and a second power consumption rate intermediate between said first power consumption rate and said third power consumption rate; and

each said computer including a local power manager determining an operating mode for itself selected from said first mode and said second mode based on processor internal activity information.

23. The computer system in claim 22, wherein said processor internal activity information comprising idle thread execution information.

24. The computer system in claim 22, wherein a transition from said first mode to said second mode is controlled locally within each said computer;

and a transition from either said first mode or said second mode to said third mode are controlled globally by said power manager.

25. The computer system in claim 24, wherein a transition from said second mode to said first mode is controlled locally within each said computer;

and a transition from said third mode to either said first mode or said second mode is controlled globally by said power manager.

26. The computer system in claim 15, wherein said third processor clock frequency is substantially zero or the third processor clock is turned off.

27. The computer system in claim 15, wherein said commands are generated and directed to said computers only when required to change an operating mode of said computers.

28. The computer system in any of claims 2 or 3, wherein said third mode is characterized by maintaining a processor core voltage to maintain processor state.

35. A system as in claim 15, wherein at least one of a processor clock frequency and a processor operating voltage is reduced in response to said indicator to thereby reduce power consumed by said processor and by said server computer.

36. A system as in claim 15, wherein the processor clock frequency is reduced in response to said indicator to thereby reduce power consumed by said processor and by said server.

37. A system as in claim 15, wherein said indicator comprises a measured decrease in server load.

38. A system as in claim 15, wherein said indicator comprises a predicted decrease in server load.

39. A system as in claim 15, wherein said indicator comprises a measured decrease in processor tasking.

40. A system as in claim 15, wherein said indicator comprises a predicted decrease in processor tasking.

41. A system as in claim 15, wherein said indicator comprises a measured decrease in communication channel bandwidth.

42. A system as in claim 15, wherein said indicator comprises predicted decrease in communication channel bandwidth.

49. A system as in claim 15, wherein the majority of content served by said server computer comprises web pages.

50. A system as in claim 15, wherein the majority of content served by said server computer comprises streaming video.

51. A system as in claim 15, wherein the majority of content served by said server computer comprises multi-media content.

52. A system as in claim 15, wherein the majority of content served by said server comprises cached data.

53. A system as in claim 15, wherein a processor operating voltage is reduced in response to said indicator to thereby reduce power consumed by said processor and by said server.

54. A system as in claim 15, wherein a predetermined combination of processor core voltage and processor frequency are selected based on predetermined rules to satisfy a processor load requirement.

56. A system as in claim 15, wherein each processor has a processor load handling capability measured in instructions per second.

57. A system as in claim 15, wherein each processor has a processor load handling capability is measured in bits served per second.

58. A system as in claim 15, wherein each processor has a processor load handling capability is measured in transactions per unit time.

59. A system as in claim 15, wherein each processor has a processor load handling capability is measured in transactions per second.

60. A system as in claim 15, wherein the predetermined rules are different for different network device types.

61. A system as in claim 3, wherein the predetermined policies include policies for identifying a non-linear relationship between processor performance and power consumption.

62. A system as in claim 3, wherein power (P) consumed by a circuit in said processor is proportional to a capacitance (C) times the product of the switching frequency of the circuit (f) and the square of the circuit operating voltage (v²) or P=Cfv².

63. A system as in claim 52, wherein switching frequency is substantially linear relative to power consumption, the load is non-linear relative to circuit performance, and the load is non-linear relative to power consumed by the circuit.

64. A system as in claim 53, wherein the circuit comprises a processor and random access memory.

65. A system as in claim 3, wherein the predetermined policy includes a policy for identifying a substantially linear relationship between processor performance and power consumption.

66. A system as in claim 3, wherein said processor provides processing for a web server and said web server has a substantially linear relationship between web page server loading an power consumed in serving said web pages.

67. A system as in claim 3, wherein a quality-of-service (QoS) is first established, and a processor performance is established based on predetermined policies that select a processor clock frequency, and a minimum processor core voltage is selected to match said selected processor clock frequency;

and wherein the established processor performance is used to control an operating mode.

68. A system as in claim 15, further including a processor core voltage control circuit receiving voltage control signals and increasing or decreasing said processor core voltage in response to said receipt.

69. A system as in claim 58, wherein said processor core voltage control circuit provides a direct-current voltage to a Vcc supply terminal of said processor.

70. A system as in claim 15, wherein said activity level indicator includes an indicator of the number of idle threads executed in said processor, and reduction of processor power consumption is initiated based on detection of the execution of an idle thread by said processor.

71. A system as in claim 60, wherein upon detection of execution of said idle thread, the processor frequency is reduced as compared to a maximum processor clock frequency.

72. A system as in claim 61, wherein said processor frequency reduction is a reduction by a factor of a power of two.

73. A system as in claim 61, wherein said processor clock frequency is reduced to zero.

74. A system as in claim 61, wherein said processor frequency is reduced to an integral multiple of a maximum processor clock frequency.

75. A system as in claim 60, wherein upon detection of execution of said idle thread, the processor frequency is reduced as compared to a maximum processor clock frequency and the processor core voltage is reduced as compared to a maximum processor core voltage.

76. A system as in claim 60, wherein said detection of execution of an idle thread initiated power reduction provides a real time adjustment to power consumption based on measured processor load requirements.

77. A system as in claim 57, wherein said QoS initiated power reduction provides a preset adjustment to power consumption based on predicted processor load requirements.

78. A system as in claim 57, wherein said QoS requirement is adjusted on at least one of a time-of-day criteria, a day-of-week criteria, a seasonal criteria, and combinations thereof.

79. A system as in claim 57, wherein said QoS requirement is adjusted based on criteria selected from the set consisting of:

time-of-day, day-of-month, day-of week, month-of year, geographic location of requester, requester identity, requester account number, and combinations thereof.

80. The computer system in claim 3, wherein said activity indicator comprises a network quality of service indicator.

81. A system as in claim 3, wherein power is conserved by controlling each computer node to enter one of said second mode or said third mode using one or more of a quality of service based predictive processor performance reduction and a activity based measured performance requirement.

82. A system as in claim 3, wherein when there is a requirement that one computer be placed in a lower power consumption mode, the computer selected for such lower power consumption is selected according to predetermined rules such that different computers are placed in lower power consumption mode each time such selection is required.

83. A system as in claim 3, wherein a computer placed in mode 3 is in a suspend state and may be woken up and placed in the first mode or the second mode by any one of a plurality of events including by a wake on LAN signal event.

84. A system as in claim 3, wherein the transition from one power consumption mode to another power consumption mode is based on a procedure implemented in software.

85. A system as in claim 3, wherein the transition from one power consumption mode to another power consumption mode is based on a procedure implemented in hardware and software.

86. A system as in claim 3, wherein when there is need to operate fewer than all the computer, the particular computer or logical group of computers that is (are) turned off or placed in a reduced power consumption mode is cycled so that over time all of the network devices experience similar operating time histories.

87. A system as in claim 3, wherein at least some of said computers include a mass storage device including a rotatable storage device.

88. A system as in claim 3, wherein said computers are configured as network server devices and a network load versus allocated network device performance profile is provided for each different type of network server device, and the performance level set for operation of said network device is established by reference to the profile.

89. A system as in claim 3, wherein at least one of said computers comprises a network server device and said activity monitoring for said network server device comprises a monitoring or either said network device load or the network device quality of service (QoS);

and wherein said monitoring is performed by said activity monitor or by a separate management computer, or both.

90. A system as in claim 3, wherein said system includes at least one temperature sensor within an enclosure holding said computers for monitoring and reporting the temperature proximate the sensor to a computers configured to monitor said temperature.

91. A system as in claim 3, wherein said system includes a plurality of temperature sensors within said enclosure reporting to one or more network devices.

92. A system as in claim 3, wherein a plurality of cooling fans are provided and operate under control of said power manager that controls each fan to provide cooling at the rate and location desired to maintain said computers within a predetermined temperature range.

93. A system as in claim 3, wherein said plurality of computers are disposed within a common enclosure and said system further comprising a plurality of temperature sensors and a plurality of cooling devices are also disposed within said enclosure, said plurality of temperature sensors communicating a temperature signal to a temperature control means and said control means adjusting the on/off status and operational parameters of the cooling units to extract heat according to predetermined rules.

94. A system as in claim 2, wherein said system further includes a plurality of power supplies and said power supplies are controlled to maintain a required power output level and operate said power supplies at a preferred efficiency.

95. A system as in claim 3, wherein the temperature of said system is moderated by motor driven cooling fans and wherein a rotational speed of said motor drive cooling is adjusted to maintain a predetermined temperature range proximate a temperature sensor.

96. A system as in claim 3, wherein the rotational speed of a motor drive cooling is adjusted to maintain a predetermined temperature range within an enclosure.

97. A system as in claim 15, wherein said activity level indicator includes an indicator of the number of idle threads executed in said processor.

98. A system as in claim 60, wherein upon detection of execution of said idle thread, the processor clock frequency is adjusted in real time so that the capability of the processor is substantially matched to the required processing capability.

99. A system as in claim 97, wherein said processor clock frequency is adjusted so that no idle threads, a predetermined number of idle threads, or a predetermined occurrence frequency of idle threads result.

100. A system as in claim 57, wherein said QoS requirement is adjusted based on criteria selected from the set consisting of:

time-of-day, day-of-month, day-of week, month-of year, geographic location of requester, requester identity, requester account number, and combinations thereof.

101. A system as in claim 3, wherein when the system includes a plurality of network devices and there is a requirement that one network device be placed in a lower power consumption mode, the network device selected for such lower power consumption is selected according to predetermined policies such that different network devices are placed in lower power consumption mode each time such selection is required.

102. A system as in claim 101, wherein said predetermined policies provide for random selection of one of the network devices.

103. A system as in claim 101, wherein said predetermined policies provide for cycling through the network devices according to some predetermined ordering.

104. A system as in claim 101, wherein said predetermined policies provide for cycling through the network devices according to some predetermined ordering in which network devices having the lowest time in service are preferentially selected for continued operation and network devices having the longest time in service are selected for reduced power operation.

105. A system as in claim 101, wherein said reduced power operation includes being powered off.

106. A system as in claim 101, wherein said reduced power operation includes being placed in a suspend mode.

107. A system as in claim 101, wherein said reduced power operation is determined according to a procedure for controlling power consumption by said system, said system having a plurality of computers operating as said network devices, each computer including at least one processor, and each computer being operable in a first mode having a first maximum performance level and a first power consumption rate, and a third mode having a third maximum performance level lower than said first maximum performance level and a third power consumption rate lower than said first power consumption rate;

said procedure comprising;

monitoring activity within said computers and identifying a level of activity for said at least one processor within said computers;

analyzing said plurality of level of activity information;

determining an operating mode for each of said computers selected from said first mode and third mode based on said analyzed activity information; and

generating commands to each of said plurality of computers directing each of said plurality of computers to operate in said determined operating mode.

108. The system in claim 107, wherein each of said computers further being operable in a second mode having a second maximum performance level intermediate between said first maximum performance level and said third maximum performance level and a second power consumption rate intermediate between said first power consumption rate and said third power consumption rate;

and said determining an operating mode further comprising determining an operating mode for each of said computers selected from said first mode, said second mode, and said third mode based on said analyzed activity information.

109. The system of claim 108, wherein a transition from said first mode to said second mode is controlled locally within each said computer;

and a transition from either said first mode or said second mode to said third mode are controlled globally by said power manager.

110. The system of claim 109, wherein a transition from said second mode to said first mode is controlled locally within each said computer;

and a transition from said third mode to either said first mode or said second mode is controlled globally by said power manager.

29. A computer system comprising:

a plurality of computers each having at least one processor and an activity monitor identifying a level of activity indicator for said at least one processor;

each of said computers being operable in;

(i) a first mode having a first maximum performance level and a first power consumption rate, and (ii) a third mode having a third maximum performance level lower than said first maximum performance level and a third power consumption rate lower than said first power consumption rate; and

a power manager;

(i) coupled to each of said computers and receiving said level of activity information from each of said plurality of computers;

(ii) analyzing said plurality of received level of activity information;

(iii) determining an operating mode for each of said computers selected from said first mode and third mode based on said analyzed activity information and predetermined policies; and

(iv) generating commands to each of said plurality of computers directing each of said plurality of computers to operate in said determined operating mode;

each of said computers further being operable in (iii) a second mode having a second maximum performance level intermediate between said first maximum performance level and said third maximum performance level and a second power consumption rate intermediate between said first power consumption rate and said third power consumption rate;

each said computer including a local power manager determining an operating mode for itself selected from said first mode and said second mode based on processor internal activity information;

a transition from said first mode to said second mode is controlled locally within each said computer, and a transition from either said first mode or said second mode to said third mode are controlled globally by said power manager; and

a transition from said second mode to said first mode is controlled locally within each said computer, and a transition from said third mode to either said first mode or said second mode is controlled globally by said power manager.

30. A computer system comprising:

a plurality of server computers each having at least one processor and an activity monitor identifying a level of activity for said at least one processor, said activity monitor comprising an activity monitor that monitors an activity selected from the set of activities consisting of;

a program application layer activity, a network layer activity, a physical layer activity, and combinations thereof;

each of said server computers being operable in;

(i) a first mode having a first maximum performance level and a first power consumption rate, (ii) a second mode having a second maximum performance level lower than said first maximum performance level and a second power consumption rate lower than said first power consumption rate, and (iii) a third mode having a third maximum performance level lower than said second maximum performance level and a third power consumption rate lower than said second power consumption rate; and

a power manager operative in a separate power manager computer;

(i) coupled to each of said server computers and receiving said level of activity information from each of said plurality of computers;

(ii) analyzing said plurality of received level of activity information;

(iii) determining an operating mode for each of said server computers selected from said first mode, second mode, and third mode based on said analyzed activity information; and

(iv) generating commands to each of said plurality of server computers directing each of said plurality of server computers to operate in said determined operating mode;

said first mode operation is characterized by operating said processor at a first processor clock frequency and a first processor core voltage, said second mode operation is characterized by operating said processor at a second processor clock frequency and a second processor core voltage, and said third mode operation is characterized by operating said processor at a third processor clock frequency and a third processor core voltage;

said second mode of operation being further characterized in that said second processor clock frequency is lower than said first processor clock frequency and said second processor core voltage is equal to or less than said first processor core voltage so that in combination consuming less power than in said first mode, and said third processor clock frequency is lower than said second processor clock frequency and said third processor core voltage is no greater than said second processor core voltage so that in combination consuming less power than in said second mode; and

a transition from said first mode to said second mode is controlled locally within each said computer; and

a transition from either said first mode or said second mode to said third mode are controlled globally by said power manager.

31. A method of operating computer system having a plurality of server computers, each server computer including at least one processor, and each computer being operable in a first mode having a first maximum performance level and a first power consumption rate, and a third mode having a third maximum performance level lower than said first maximum performance level and a third power consumption rate lower than said first power consumption rate;

said method comprising;

monitoring activity within said computers and identifying a level of activity for said at least one processor within said computers;

analyzing said plurality of level of activity information;

determining an operating mode for each of said computers selected from said first mode and third mode based on said analyzed activity information; and

generating commands to each of said plurality of computers directing each of said plurality of computers to operate in said determined operating mode.

32. The method in claim 31, wherein each of said computers further being operable in a second mode having a second maximum performance level intermediate between said first maximum performance level and said third maximum performance level and a second power consumption rate intermediate between said first power consumption rate and said third power consumption rate;

and said determining an operating mode further comprising determining an operating mode for each of said computers selected from said first mode, said second mode, and said third mode based on said analyzed activity information.

33. The method of claim 32, wherein a transition from said first mode to said second mode is controlled locally within each said computer;

and a transition from either said first mode or said second mode to said third mode are controlled globally by said power manager.

34. The method of claim 33, wherein a transition from said second mode to said first mode is controlled locally within each said computer;

and a transition from said third mode to either said first mode or said second mode is controlled globally by said power manager.

43. A system as in claim 32, wherein said predicted decrease in server load is a prediction based at least in part on time of day.

44. A system as in claim 32, wherein said predicted decrease in server load is a prediction based at least in part on a quality of service requirement.

45. A system as in claim 32, wherein said predicted decrease in processor tasking is a prediction based at least in part on time of day.

46. A system as in claim 32, wherein said predicted decrease in processor tasking is a prediction based at least in part type of content to be processed.

47. A system as in claim 32, wherein said predicted decrease in server loading is a prediction based at least in part type of content to be served.

48. A system as in claim 32, wherein the manner of said prediction is further based on the content served by the server.

55. A system as in claim 46, wherein said predetermined rules for selecting said predetermined combination of processor core voltage and processor frequency comprise a look-up-table (LUT) identifying processor frequency and processor core voltage with processor load handling capability.

111. In a server farm comprising a multiplicity of computer systems operating as content servers, a method of operating said servers, each server computer including at least one processor, and each computer being operable in a first mode having a first maximum performance level and a first power consumption rate, and a third mode having a third maximum performance level lower than said first maximum performance level and a third power consumption rate lower than said first power consumption rate;

said method comprising;

monitoring activity within each said computer server and identifying a level of activity for said at least one processor within said server computer;

analyzing said plurality of level of activity information;

determining an operating mode for each of said computers selected from said first mode and third mode based on said analyzed activity information; and

generating commands to each of said multiplicity of server computers directing each of said plurality of computers to operate in said determined operating mode.

112. A network server computer system comprising:

a plurality of network server computers configured as network device servers each having at least one processor and a network layer activity monitor that monitors TCP/IP protocol data packets identifying a level of activity indicator for each said at least one processor;

each of said network device server computers including a network load versus allocated network device performance profile and the performance level set for operation of each said network device server being established by reference to the performance profile;

each of said network device server computers being operable in;

(i) a first mode having a first maximum performance level and a first power consumption rate, (ii) a third mode having a third maximum performance level lower than said first maximum performance level and a third power consumption rate lower than said first power consumption rate, and (iii) a second mode having a second maximum performance level intermediate between said first maximum performance level and said third maximum performance level and a second power consumption rate intermediate between said first power consumption rate and said third power consumption rate;

a power manager selected as one of said plurality of computers being designated as a master providing said power manager and (i) coupled to each of said computers and receiving said level of activity information from each of said plurality of computers;

(ii) analyzing said plurality of received level of activity information;

(iii) determining an operating mode for each of said computers selected from said first mode, said second mode, and said third mode based on said analyzed activity information and predetermined policies; and

including incrementally lowering processor performance by said power manager until data packets start dropping indicating that processor performance is at the limit of adequacy and then increasing the processor performance by a specified increment to act as a safety margin to provide reliable communication of the packets; and

(iv) generating commands to each of said plurality of computers directing each of said plurality of computers to operate in said determined operating mode;

said first mode operation being characterized by operating said processor at a first processor clock frequency and a first processor core voltage, said second mode operation is characterized by operating said processor at a second processor clock frequency and a second processor core voltage, and said third mode operation is characterized by operating said processor at a third processor clock frequency and a third processor core voltage;

said second mode of operation being characterized in that said second processor clock frequency and said second processor core voltage in combination consuming less power than said first processor clock frequency and said first processor core voltage in combination, and said third processor clock frequency and said third processor core voltage in combination consuming less power than said second processor clock frequency and said second processor core voltage in combination;

said processor core voltage and processor frequency being selected in combination with reference to a stored look-up-table identifying processor frequency and processor core voltage with processor load handling capability for serving content, and network devices server computer loading and quality-of-service being measured for the plurality of physical network devices organized as a single logical network device;

said level of activity indicator comprises an indicator based on at least one of a predicted decrease in server load based at least in part on a quality of service requirement and the type of content to be served, a predicted decrease in processor tasking based at least in part on a quality of service requirement and the type of content to be served, a predicted decrease in communication channel bandwidth based at least in part on a quality of service requirement and the type of content to be served;

a quality-of-service (QoS) is first established, and a processor performance is established based on predetermined policies that select a processor clock frequency, and a minimum processor core voltage is selected to match said selected processor clock frequency; and

said quality of service requirement is adjusted based on criteria selected from the set consisting of;

time-of-day, day-of-month, day-of week, month-of year, geographic location of requester, requester identity, requester account number, and combinations thereof.

113. The system in claim 112, wherein said activity monitor comprises an activity monitor that monitors an activity selected from the set of activities consisting of:

a program application layer activity, a network layer activity, a physical layer activity, and combinations thereof.

114. The system in claim 113, wherein at the physical level the number of processor idle threads executed within a predetermined period of time are measured to determine processor loading and the processor performance is adjusted to by altering the operating mode to substantially match the level of processor loading.

115. The system in claim 114, wherein said application layer activity monitor comprises monitoring use of a TCP/IP port address within said computers, said monitoring including counting or measuring a number of times a specific port address is being requested within a predetermined period of time, and in response to that counting or measurement, placing a sufficient amount of computer performance to meet the performance requirement for each application requesting the port address.

116. The system in claim 112, wherein each said computer including a local power manager determining an operating mode for itself selected from said first mode and said second mode based on processor internal activity information.

117. The system in claim 116, wherein said processor internal activity information comprising idle thread execution information.

118. The system in claim 117, wherein a transition from said first mode to said second mode is controlled locally within each said computer;

and a transition from either said first mode or said second mode to said third mode are controlled globally by said power manager.

119. The system in claim 112, wherein when there is need to operate fewer than all the plurality of computers, the particular computer or logical group of computers that is (are) turned off or placed in a reduced power consumption mode is cycled so that over time all of the network devices experience similar operating time histories.