Method of correlating power in a data center by fitting a function to a plurality of pairs of actual power draw values and estimated power draw values determined from monitored CPU utilization of a statistical sample of computers in the data center

• US 9,009,500 B1
• Filed: 01/18/2012
• Issued: 04/14/2015
• Est. Priority Date: 01/18/2012
• Status: Expired due to Fees

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Fan, Xiaobo, Heath, Taliver Brooks, Hennecke, Mark D.
• Primary Examiner(s): Abbaszadeh, Jaweed A
• Assistant Examiner(s): JOHNSON, TERRELL S
• Application Number:
  US13/352,516
• Time in Patent Office: 1,182 Days
• Field of Search: 713/300, 713/321, 713/322
• US Class Current: 713/300
• CPC Class Codes: G06F 1/00 Details not covered by groups G06F3/00 - G06F13/00 and G06F21/00 architectures of general purpose stored program computers G06F15/76
• CPC Class Codes: G06F 1/26 Power supply means, e.g. regulation thereof for memories G11C
• CPC Class Codes: G06F 9/50 Allocation of resources, e.g. of the central processing unit [CPU]
• CPC Class Codes: G16Z 99/00 Subject matter not provided for in other main groups of this subclass

Abstract

The subject matter of this specification can be embodied in, among other things, a method that includes supplying power to a portion of a data center through a power distribution line. Utilization of a statistically significant sample of the computers is monitored, and an estimated individual power draw for each of the computers based on the utilization is calculated. An estimated total power draw is calculated for different times from the estimated individual power draws to generate a collection of estimated total power draw values for the different times. Actual power draw is monitored at the power distribution line and a collection of actual power draw values is generated. A function is fitted to pairs of actual power draw values and estimated power draw values, each pair comprising an actual draw value and an estimated draw value for the same time, and the function is then stored.

Claims

1. A method of correlating power in a data center, comprising:

supplying power to a portion of a data center through a power distribution line, the portion of the data center including a plurality of computers that draw power through a circuit breaker;

monitoring CPU utilization of a statistical sample of the plurality of computers, the statistical sample being a statistically significant number of the plurality of computers that is less than all of the plurality of computers;

calculating an estimated individual power draw for each of the statistical sample of the plurality of computers based on the utilization;

calculating an estimated total power draw for each of a plurality of different times from the estimated individual power draw of each of the statistical sample of the plurality of computers to generate a plurality of estimated total power draw values for the plurality of different times;

monitoring actual power draw at the power distribution line by the portion of the data center and generating a plurality of actual power draw values for the plurality of different times;

fitting a function to a plurality of pairs of actual power draw values and estimated power draw values, each pair of the plurality of pairs comprising an actual draw value and an estimated draw value for the same time; and

storing the function.

2. The method of claim 1, wherein calculating the estimated individual power draw comprises storing data representing a computer configuration for each of the plurality of computers, and the computer configuration is an input to a function relating utilization to estimated individual power.

3. The method of claim 2, wherein the computer configuration includes one or more of processor speed, amount of memory or number of disk drives.

4. The method of claim 1, wherein calculating the estimated total power draw comprises summing the estimated individual power draw of each of the statistical sample of the plurality of computers.

5. The method of claim 1, wherein the function is a linear function.

6. The method of claim 1, further comprising:

receiving a maximum power draw value for the portion of the data center;

calculating an estimated current individual power draw value for each of the statistical sample of the plurality of computers based on the utilization;

calculating an estimated current total power draw value from the estimated current individual power draw value of each of the statistical sample of the plurality of computers;

at least one of

calculating an adjusted maximum power draw value from the maximum power draw value and the function, and comparing the estimated current total power draw value to the adjusted maximum power draw value, or

calculating an adjusted estimated current total power draw value from the estimated current total power draw value and the function, and comparing the adjusted estimated total power draw value to the maximum power draw value; and

adjusting operation of at least one computer in the plurality of computers based on the comparison.

7. The method of claim 6, wherein adjusting operation comprises one or more of adjusting job allocation, adjusting job scheduling or adjusting central processing unit execution frequency.

8. The method of claim 6, comprising calculating the adjusted maximum power draw value from the maximum power draw value and the function, and comparing the estimated current total power draw value to the adjusted maximum power draw value.

9. The method of claim 8, wherein adjusting operation comprises adjusting a multiplicity of computers in the plurality of computers to cause the estimated current total power draw value to be closer to the adjusted maximum power draw value.

10. The method of claim 6, comprising calculating the adjusted estimated current total power draw value from the estimated current total power draw value and the function, and comparing the adjusted estimated total power draw value to the maximum power draw value.

11. The method of claim 10, wherein adjusting operation comprises adjusting a multiplicity of computers in the plurality of computers to cause the adjusted estimated total power draw value to be closer to the maximum power draw value.

12. The method of claim 1, wherein calculating an estimated individual power draw for a computer of the plurality of computers comprises measuring utilization of the computer at a plurality of utilization measurement times to generate a plurality of utilization values for the computer, there being a greater number of utilization measurement times than different times such that there are a greater number of utilization values for the computer than different times.

13. The method of claim 12, further comprising associating each of the plurality of utilization values with one of the plurality of different times.

14. The method of claim 13, wherein associating includes comparing a utilization measurement time for a utilization value with a midpoint between two adjacent different times.

15. The method of claim 13, wherein there is a plurality of utilization value times for each of plurality of different times.

16. The method of claim 15, wherein calculating the estimated total power draw for one of the different times comprises weighting each estimated individual power draw in the plurality of estimated power draw values based on a difference between a value estimating time for the estimate value and the different time.

17. The method of claim 12, further comprising synchronizing time stamps of the different times and the utilization measurement times.

18. The method of claim 1, wherein calculating an estimated total power draw occurs at a higher frequency than monitoring actual power draw.

19. The method of claim 1, wherein the plurality of computers comprises a population of computers in a datacenter.

20. A data center, comprising:

a power distribution line;

a plurality of computers that draw power from the power distribution line through a circuit breaker, wherein each computer of a statistical sample of the plurality of computers is configured to generate a measurement of CPU utilization of the computer, the sample being a statistically significant number of the plurality of computers that is less than all of the plurality of computers;

a processor configured to

receive the measurement of CPU utilization from the statistical sample of the plurality of computer,

calculate an estimated individual power draw for each of the statistical sample of the plurality of computers based on the utilization,

calculate an estimated total power draw for each of a plurality of different times from the estimated individual power draw of each of the statistical sample of the plurality of computers to generate a plurality of estimated total power draw values for the plurality of different times,

receive a measurement of an actual power draw at the power distribution line by the portion of the data center and generate a plurality of actual power draw values for the plurality of different times,

fit a function to a plurality of pairs of actual power draw values and estimated power draw values, each pair of the plurality of pairs comprising an actual draw value and an estimated draw value for the same time, and

store the function.

21. A non-transitory computer readable medium storing instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising:

receiving the measurement of CPU utilization from a statistical sample of a plurality of computers, the statistical sample being a statistically significant number of the plurality of computers that is less than all of the plurality of computers,

calculate an estimated individual power draw for each of the statistical sample of the plurality of computers based on the utilization,

calculate an estimated total power draw for each of a plurality of different times from the estimated individual power draw of each of the statistical sample of the plurality of computers to generate a plurality of estimated total power draw values for the plurality of different times,

receive a measurement of an actual power draw at the power distribution line by the portion of the data center and generate a plurality of actual power draw values for the plurality of different times,

fit a function to a plurality of pairs of actual power draw values and estimated power draw values, each pair of the plurality of pairs comprising an actual draw value and an estimated draw value for the same time, and

store the function.

22. The computer readable medium of claim 21, wherein the plurality of computers comprises a population of computers in a datacenter.