Workload placement based on thermal considerations

• US 20060047808A1
• Filed: 08/31/2004
• Published: 03/02/2006
• Est. Priority Date: 08/31/2004
• Status: Active Grant

• Current Assignee: Hewlett Packard Enterprise Development LP (Hewlett-Packard Enterprise Company)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Bash, Cullen E., Sharma, Ratnesh K., Patel, Chandrakant D., Friedrich, Richard J.
• Granted Patent: US 7,447,920 B2
• Time in Patent Office: Days
• US Class Current: 709/224
• CPC Class Codes: G06F 2209/5022 Workload threshold
• CPC Class Codes: G06F 9/505 considering the load
• CPC Class Codes: G06F 9/5061 Partitioning or combining of resources
• CPC Class Codes: G06F 9/5094 where the allocation takes into account power or heat criteria power management in computers in general G06F1/3203; thermal management in computers in general G06F1/206
• CPC Class Codes: H05K 7/20836 Thermal management, e.g. server temperature control
• CPC Class Codes: Y02D 10/00 Energy efficient computing, e.g. low power processors, power management or thermal management

Abstract

A method of workload placement among servers includes receipt of a workload request. The method also includes selection of a group of servers from a set of servers capable of performing the requested workload. In addition, server workload indexes, which are ratios of temperature rises due to re-circulation load for the servers in the group of servers to temperature rises due to re-circulation load for the set of servers, of the servers in the group of servers are calculated. Moreover, the workload is placed on one or more of the servers in the group of servers based upon the server workload indexes of the servers in the group of servers.

Claims

1. A method of workload placement among servers, said method comprising:

receiving a workload request;

selecting a group of servers from a set of servers capable of performing the requested workload;

calculating server workload indexes of the servers in the group of servers, wherein the server workload indexes comprise ratios of temperature rises due to re-circulation load for the servers in the group of servers to temperature rises due to re-circulation load for the set of servers; and

placing the workload on one or more of the servers in the group of servers based upon the server workload indexes of the servers in the group of servers.

2. The method according to claim 1, further comprising:

collecting inlet and outlet temperatures of the servers in the group of servers;

collecting supply air temperatures for one or more CRAC units;

calculating supply heat indexes (φ

) of the servers in the group of servers based upon the collected inlet and outlet temperatures and the supply air temperatures;

calculating thermal multipliers (θ

) for the servers in the group of servers, wherein the thermal multipliers are operable to estimate the relative abilities of the servers in the group of servers to accept new workloads compared to the servers in the set of servers; and

wherein the step of calculating server workload indexes comprises using the supply heat indexes to calculate the server workload indexes.

3. The method according to claim 2, wherein the step of calculating supply heat indexes comprises calculating supply heat indexes (φ

) of the servers in the group of servers based upon the following equation;

4. The method according to claim 2, wherein the step of calculating thermal multipliers (θ

) comprises calculating thermal multipliers (θ

) of the servers in the group of servers based upon the following equation;

5. The method according to claim 2, wherein the step of calculating server workload indexes (SWI) comprises calculating server workload indexes (SWI) of the servers in the group of servers based upon the following equation:

6. The method according to claim 1, further comprising:

determining which of the servers in the group of servers have server workload indexes that fall below a predetermined maximum threshold value;

withdrawing the servers from the group of servers having server workload indexes that exceed the predetermined maximum threshold value;

estimating available states (Δ

m) to perform the workload in the servers remaining in the group of servers;

determining which of the remaining servers have the available states; and

wherein the step of placing the workload on one or more of the servers comprises placing the workload on one or more servers having the available states.

7. The method according to claim 6, wherein the step of estimating available states comprising estimating available states (Δ

m) based upon the following equation;

Δ

m_i,j,k=int[[1−

SWI](N−

m)]_i,j,k, where the subscript i,j,k denotes the k^th server in the j^th rack of the i^th row, SWI is the server workload index, N is the number of states in a server, and m is the number of states being utilized by the server.

8. The method according to claim 6, further comprising:

calculating placement factors for the one or more servers having the available states, wherein the placement factors (X) for each of the one or more servers is determined according to the following equation;

X_i,j,k=SHI_i,j,k−

SWI_i,j,k, where the subscript i,j,k denotes the k^th server in the j^th rack of the i^th row, SHI is a supply heat index and SWI is the server workload index; and

wherein the step of placing the workload further comprises placing the workload on one or more of the servers having relatively high positive values of X.

9. The method according to claim 6, wherein the step of placing the workload further comprises placing a portion of the workload on one or more servers in close proximity to one another, said method further comprising:

assessing an impact on thermal conditions based upon the workload placement; and

placing another portion of the workload on the one or more servers in close proximity to one another in response to the impact on thermal conditions being acceptable.

10. The method according to claim 6, wherein the step of placing the workload further comprises placing a portion of the workload on one or more servers in close proximity to one another, said method further comprising:

assessing an impact on thermal conditions based upon the workload placement;

withdrawing the portion of the workload from the one or more servers;

selecting another group of servers;

calculating server workload indexes of the servers in the another group of servers; and

placing the workload on one or more of the servers in the another group of servers based upon the server workload indexes of the servers in the another group of servers.

11. The method according to claim 6, further comprising:

considering additional factors in selecting the workload placement prior to the step of placing the workload, wherein the additional factors comprises one or more of speeds of processors contained in the servers in the group of servers, security levels of the servers in the group of servers, and terms of one or more service level agreements.

12. The method according to claim 6, further comprising:

collecting inlet and outlet temperatures of the servers in the group of servers;

collecting supply air temperatures for one or more CRAC units;

calculating supply heat indexes of the servers in the group of servers based upon the collected inlet and outlet temperatures and the supply air temperatures;

plotting the supply heat indexes against the inlet temperatures of the servers in the group of servers at a plurality of flow field settings;

determining a relationship of the plotted supply heat indexes and inlet temperatures; and

wherein the step of placing the workload comprises placing the workload on one or more of the servers in the group of servers based upon the relationship of the plotted supply heat indexes and inlet temperatures of the servers in the group of servers.

13. The method according to claim 12, further comprising:

determining which of the servers in the group of servers have plotted supply heat indexes and inlet temperatures that have nearly linear relationships prior to the step of placing the workload.

14. The method according to claim 1, further comprising:

categorizing the servers in the group of servers based upon the server workload indexes of the servers into a plurality of bins;

selecting the servers categorized in a predetermined bin;

withdrawing the servers categorized in a bin different from the predetermined bin;

estimating available states (Δ

m) to perform the workload in the servers categorized in the predetermined bin, wherein the available states (Δ

m) are estimated according to the following equation;

Δ

m_i,j,k=int[[1−

SWI](N−

m)]_i,j,k, where the subscript i,j,k denotes the k^th server in the j^th rack of the i^th row, SWI is the server workload index, N is the number of states in a server, and m is the number of states being utilized by the server;

determining which of the servers categorized in the predetermined bin have the available states; and

wherein the step of placing the workload on one or more of the servers comprises placing the workload on one or more servers having the available states.

15. The method according to claim 14, further comprising:

calculating placement factors for the one or more servers having the available states, wherein the placement factors (X) for each of the one or more servers is determined according to the following equation;

X_i,j,k=SHI_i,j,k−

SWI_i,j,k, where the subscript i,j,k denotes the k^th server in the j^th rack of the i^th row, SHI is a supply heat index and SWI is the server workload index; and

wherein the step of placing the workload further comprises placing the workload on one or more of the servers having relatively high positive values of X.

16. The method according to claim 15, wherein the step of placing the workload further comprises placing a portion of the workload on one or more servers in close proximity to one another, said method further comprising:

assessing an impact on thermal conditions based upon the workload placement; and

placing another portion of the workload on the one or more servers in close proximity to one another in response to the impact on thermal conditions being acceptable.

17. The method according to claim 15, wherein the step of placing the workload further comprises placing a portion of the workload on one or more servers in close proximity to one another, said method further comprising:

assessing an impact on thermal conditions based upon the workload placement;

withdrawing the portion of the workload from the one or more servers;

selecting another group of servers;

calculating server workload indexes of the servers in the another group of servers; and

placing the workload on one or more of the servers in the another group of servers based upon the server workload indexes of the servers in the another group of servers.

18. The method according to claim 1, further comprising:

determining whether thermal management conditions are acceptable in response to the step of placing the workload on one or more servers in the group of servers;

withdrawing the workload from the one or more servers in response to the conditions being unacceptable;

withdrawing the one or more servers from the group of servers to form a second group of servers;

calculating server workload indexes of the servers in the second group of servers; and

placing the workload on one or more of the servers in the second group of servers based upon the server workload indexes of the servers in the group of servers.

19. The method according to claim 18, wherein the step of determining whether the thermal management conditions are acceptable comprises calculating supply heat indexes of the set of servers following the step of placing the workload.

20. A data center having a system for workload placement among a plurality of servers, said data center comprising:

inlet temperature sensors configured to detect inlet temperatures of the plurality of servers;

outlet temperature sensors configured to detect outlet temperatures of the plurality of servers;

one or more CRAC units configured to deliver cooled air to the plurality of servers;

one or more supply air temperature sensors configured to detect temperatures of the cooled air supplied by the one or more CRAC units; and

a resource manager configured to receive the detected inlet, outlet and supply air temperatures and to calculate server workload indexes of the servers based upon the detected inlet, outlet, and supply air temperatures, and wherein the resource manager is further configured to place the workload on one or more of the servers based upon the server workload indexes of the servers.

21. The data center according to claim 20, wherein the server workload indexes comprise ratios of temperature rises due to re-circulation load for the servers in the group of servers to temperature rises due to re-circulation load for a set of servers, which includes the group of servers.

22. The data center according to claim 20, wherein the resource manager is configured to calculate supply heat indexes of the plurality of servers and to calculate thermal multipliers for the plurality of servers, wherein the thermal multipliers are operable to estimate the relative abilities of the plurality of servers to accept new workloads compared to other servers.

23. The data center according to claim 22, wherein the supply heat indexes (φ

) are calculated according to the following equation;

24. The data center according to claim 22, wherein the resource manager is further configured to plot the supply heat indexes against the inlet temperatures of the plurality of servers at a plurality of flow field settings, to determine a relationship of the plotted supply heat indexes and inlet temperatures, and to place the workload on one or more of the plurality of servers based upon the linearity of the relationships.

25. The data center according to claim 20, wherein the resource manager is further configured to select a group of servers to perform the workload through at least one of determining which of the plurality of servers have server workload indexes that fall below a predetermined maximum threshold value and withdrawing the servers having server workload indexes that exceed the predetermined maximum threshold value, and categorizing the plurality of servers into groups based upon the server workload indexes of the plurality of servers, selecting the servers categorized in a predetermined group, and withdrawing the servers categorized in a group different from the predetermined group.

26. The data center according to claim 25, wherein the resource manager is further configured to estimate available states of the servers in the group of servers, to determine which of the servers have available states, and to place the workload on one or more of the servers having the available states, wherein the available states (Δ

m) are estimated according to following equation;

Δ

m_i,j,k=int[[1−

SWI](N−

m)]_i,j,k where the subscript i,j, k denotes the k^th server in the j^th rack of the i^th row, SWI is the server workload index, N is the number of states in a server, and m is the number of states being utilized by the server.

27. The data center according to claim 26, wherein the resource manager is further configured to consider additional factors in selecting the workload placement, wherein the additional factors comprises one or more of speeds of processors contained in the plurality of servers, security levels of the plurality of servers, and terms of one or more service level agreements.

28. The data center according to claim 26, wherein the resource manager is further configured to calculate placement factors for the servers in the group of servers having the available states, wherein the placement factors (X) for each of the one or more servers is determined according to the following equation:

X_i,j,k=SHI_i,j,k−

SWI_i,j,k, where the subscript i,j,k denotes the k^th server in the j^th rack of the i^th row, SHI is a supply heat index and SWI is the server workload index; and

wherein the step of placing the workload further comprises placing the workload on one or more of the servers having relatively high positive values of X.

29. The data center according to claim 20, wherein the resource manager is further configured to determine whether thermal management conditions are acceptable in response to the workload placement, to withdraw the workload from the one or more of the plurality of servers in response to the conditions being unacceptable, to withdraw the one or more of the plurality of servers from consideration to form a second group of servers, to calculate server workload indexes for the servers in the second group of servers, and to place the workload on one or more of the servers in the second group of servers based upon the server workload indexes of the servers in the group of servers.

30. The data center according to claim 29, wherein the determination of whether the thermal management conditions are acceptable comprises the calculation of supply heat indexes of the plurality of servers following placement of the workload.

31. A system for workload placement among a plurality of servers, said system comprising:

means for receiving a workload request;

means for selecting a group of servers from a set of servers capable of performing the requested workload;

means for collecting inlet and outlet temperatures of the servers in the group of servers;

means for collecting supply air temperatures for one or more CRAC units;

means for calculating supply heat indexes (φ

) of the servers in the group of servers based upon the collected inlet and outlet temperatures and the supply air temperatures;

means for calculating thermal multipliers (θ

) for the servers in the group of servers, wherein the thermal multipliers are operable to estimate the relative abilities of the servers in the group of servers to accept new workloads compared to the servers in the set of servers

means for calculating server workload indexes (SWI) of the servers in the group of servers, wherein the server workload indexes are calculated according to the supply heat indexes and the thermal multipliers of the servers in the group of servers;

means for estimating available states to perform the workload in the servers of the group of servers having server workload indexes that fall below a predetermine maximum threshold value; and

means for placing the workload on one or more of the servers in the group of servers having the available states.

32. The system according to claim 31, wherein the means for calculating supply heat indexes (φ

) of the servers in the group of servers comprises means for calculating the following equation;

33. The system according to claim 31, wherein the means for estimating available states (Δ

m) comprises means for calculating the following equation;

Δ

m_i,j,k=int[[1−

SWI](N−

m)]_i,j,k, where the subscript i,j,k denotes the k^th server in the j^th rack of the i^th row, N is the number of states in a server, and m is the number of states being utilized by the server.

34. A computer readable storage medium on which is embedded one or more computer programs, said one or more computer programs implementing a method of workload placement among a plurality of servers, said one or more computer programs comprising a set of instructions for:

receiving a workload request;

selecting a group of servers from a set of servers capable of performing the requested workload;

collecting inlet and outlet temperatures of the servers in the group of servers;

collecting supply air temperatures for one or more CRAC units;

calculating supply heat indexes (φ

) of the servers in the group of servers based upon the collected inlet and outlet temperatures and the supply air temperatures;

calculating thermal multipliers (θ

) for the servers in the group of servers, wherein the thermal multipliers are operable to estimate the relative abilities of the servers in the group of servers to accept new workloads compared to the servers in the set of servers

calculating server workload indexes (SWI) of the servers in the group of servers, wherein the server workload indexes are calculated according to the supply heat indexes and the thermal multipliers of the servers in the group of servers;

estimating available states to perform the workload in the servers of the group of servers having server workload indexes that fall below a predetermine maximum threshold value; and

placing the workload on one or more of the servers in the group of servers having the available states.

35. The computer readable storage medium according to claim 34, said one or more computer programs further comprising a set of instructions for:

calculating the supply heat indexes (φ

) of the servers in the group of servers based upon the following equation;

$\mathrm{\varphi <br><br>}={\left[\frac{\mathrm{\delta <br><br>}<br><br>\text{\hspace{1em}<br><br>}<br><br>Q}{Q+\mathrm{\delta <br><br>}<br><br>\text{\hspace{1em}<br><br>}<br><br>Q}\right]}_{i,j,k}={\left[\frac{T_{i<br><br>\text{\hspace{1em}<br><br>}<br><br>n}-T_{\mathrm{ref}}}{T_{\mathrm{out}}-T_{\mathrm{ref}}}\right]}_{i,j,k},$

where δ

Q is the heat load due to hot air infiltration, Q is the actual heat load, the subscript i,j,k denotes the k^th server in the j^th rack of the i^th row, T_in denotes the inlet temperature and T_out denotes the outlet temperature of each server in the group of servers, and T_ref denotes the supply air temperatures of the one or more CRAC units;

calculating the thermal multipliers (θ

) means based upon the following equation;

${\mathrm{\theta <br><br>}}_{i,j,k}={\left(\frac{{\stackrel{\_}{T}}_{\mathrm{out}}-T_{\mathrm{ref}}}{T_{\mathrm{out}}-T_{\mathrm{ref}}}\right)}_{i,j,k},$

where {overscore (T)}_out denotes an average server exhaust temperature of the set of servers, and T_ref denotes the supply air temperatures of the one or more CRAC units; and

calculating the server workload indexes (SWI) based upon the following equation;

${\left(\mathrm{SWI}\right)}_{i,j,k}=\frac{\mathrm{\varphi <br><br>}}{\mathrm{\theta <br><br>}}={\left(\frac{\left(T_{i<br><br>\text{\hspace{1em}<br><br>}<br><br>n}-T_{\mathrm{ref}}\right)}{\left({\stackrel{\_}{T}}_{\mathrm{out}}-T_{\mathrm{ref}}\right)}\right)}_{i,j,k}.$

36. The computer readable storage medium according to claim 34, said one or more computer programs further comprising a set of instructions for:

estimating the available states (Δ

m) based upon the following equation;

Δ

m_i,j,k=int[[1−

SWI](N−

m)]_i,j,k, where the subscript i,j,k denotes the k^th server in the j^th rack of the i^th row, N is the number of states in a server, and m is the number of states being utilized by the server.