Data center design

• US 10,339,227 B1
• Filed: 03/31/2008
• Issued: 07/02/2019
• Est. Priority Date: 06/08/2007
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Carlson, Andrew B., Clidaras, Jimmy, Hamburgen, William, Weber, Wolf-Dietrich, Fan, Xiaobo, Barroso, Luiz Andre
• Primary Examiner(s): Chad, Aniss
• Application Number:
  US12/060,211
• Time in Patent Office: 4,110 Days
• Field of Search: 703 1
• CPC Class Codes: G06F 1/00 Details not covered by groups G06F3/00 - G06F13/00 and G06F21/00 architectures of general purpose stored program computers G06F15/76
• CPC Class Codes: G06F 1/26 Power supply means, e.g. regulation thereof for memories G11C
• CPC Class Codes: G06F 1/3206 Monitoring of events, devices or parameters that trigger a change in power modality
• CPC Class Codes: G06F 2119/08 Thermal analysis or thermal optimisation
• CPC Class Codes: G06F 30/13 Architectural design, e.g. computer-aided architectural design [CAAD] related to design of buildings, bridges, landscapes, production plants or roads
• CPC Class Codes: G06F 30/20 Design optimisation, verification or simulation optimisation, verification or simulation of circuit designs G06F30/30
• CPC Class Codes: G06F 9/5094 where the allocation takes into account power or heat criteria power management in computers in general G06F1/3203; thermal management in computers in general G06F1/206
• CPC Class Codes: H05K 7/20836 Thermal management, e.g. server temperature control
• CPC Class Codes: Y02D 10/00 Energy efficient computing, e.g. low power processors, power management or thermal management

Abstract

A method of designing a data center can include determining a design power density, determining an oversubscription ratio, and determining a spatial layout of the data center using the design power density and the oversubscription ratio. The design power density can be in units of power per unit length.

Claims

1. A method for constructing a data center, comprising:

determining, using a computer system, a design power density using inputs that include (a) a platform roadmap that provides a projection of a peak power draw of a group of computers in the data center; and

(b) at least one facility component restraint defining an upper bound of the design power density, wherein the design power density is in units of power per unit length;

determining, using the computer system, an oversubscription ratio that defines a ratio of the design power density to a representative power density of the group of computers;

determining, using the computer system and based on the oversubscription ratio, that the peak power draw of the group of computers exceeds a power capacity of the data center and an average power draw of the group of computers is less than the power capacity of the data center;

determining, using the computer system, a spatial layout for the group of computers for deployment in the data center of the data center using the design power density and the oversubscription ratio, wherein determining the spatial layout includes determining a total length of one or more rows of racks of computers and determining a layout of power taps; and

deploying the group of computers in the data center in the determined spatial layout.

2. The method of claim 1, further comprising constructing the data center having the spatial layout comprising:

determining that a peak power draw of a part of the group of computers remains within the power capacity of the data center;

deploying the part of the group of the computers in the data center;

collecting measurements of actual power draw;

revising the projection of the peak power draw based on the measurements of actual power draw; and

deploying the group of computers in the data center such that the peak power draw of the group of computers exceeds the power capacity of the data center and the average power draw of the group of computers is less than the power capacity of the data center.

3. The method of claim 1, wherein determining the total length includes dividing a power budget by the design power density and multiplying by the oversubscription ratio.

4. The method of claim 1, wherein determining the layout of power taps includes selecting a number of taps per unit length and an amperage of the taps such that the design power density is not exceeded.

5. The method of claim 1, wherein determining the design power density includes generating a projected power draw of at least one computer in the group of computers.

6. The method of claim 1, wherein determining the design power density includes determining a number of computers per unit length along a row.

7. The method of claim 1, wherein determining the oversubscription ratio includes calculating a ratio of the peak power draw of the group of computers to the average power draw of the group of computers.

8. The method of claim 7, wherein the peak power draw comprises an expected peak power draw of a computer to be installed in the data center at a later point in time relative to an initial operation of the data center.

9. The method of claim 8, further comprising determining the peak power draw by determining the components in the computer, determining peak power usage values for the components, and adding the peak power usage values together.

10. The method of claim 7, wherein the average power draw is an average of power draw across the group of computers.

11. The method of claim 7, further comprising determining the average power draw by determining power usage distributions for the group of computers and calculating a peak power draw per computer for the group of computers from the power usage distributions.

12. The method of claim 1, further comprising calculating a heat dissipation from the design power density.

13. The method of claim 12, further comprising constructing the data center with a cooling system with sufficient capacity to handle the heat dissipation.

14. The method of claim 1, further comprising constructing the data center having a power carrying-capability meeting the design power density.

15. The method of claim 1, further comprising determining a number of computing systems that can be run in the data center.

16. The method of claim 15, wherein determining the number of computing systems that can be run in the data center includes dividing a total power available by the average power draw.

17. The method of claim 1, wherein the group of computers comprises a first group of computers, the method further comprising installing a second group of computers in the data center, wherein the second group of computers is unequal in quantity to the first group of computers.

18. The method of claim 17, wherein the second group of computers includes at least 1000 computers.

19. The method of claim 17, wherein the second group of computers includes at least 5000 computers.

20. The method of claim 17, further comprising:

determining, using the computer system, a projection of a peak power draw of the second group of computers in the data center;

determining, using the computer system, a revised design power density based at least in part on the projection of a peak power draw of the second group of computers in the data center; and

determining, using the computer system, a revised spatial layout of the data center using the revised design power density and the oversubscription ratio.

21. The method of claim 1, wherein the group of computers comprises one group of a plurality of groups of computers within a plurality of computers.

22. The method of claim 1, wherein the upper bound is less than a maximum power capacity of the system.

23. The method of claim 1, wherein the representative power density of the group of computers comprises an average power density of the group of computers.

24. The method of claim 1, further comprising:

deploying a first set of the group of computers based on the spatial layout;

determining a revised oversubscription ratio based on actual power draw of the first set of the group of computers deployed based on the spatial layout; and

determining, using the computer system, a second spatial layout for the group of computers of the data center using the design power density and the revised oversubscription ratio.

25. The method of claim 24, further comprising increasing the number of computers in the group of computers based on the revised oversubscription ratio.

26. The method of claim 1, wherein determining the spatial layout comprises determining a hierarchical power distribution system comprising a rack level, and wherein a total peak power draw of a rack will not exceed a power capacity of the rack.

27. The method of claim 1, wherein

determining the spatial layout comprises determining a hierarchical power distribution system comprising a lower level and a higher level such that a total peak power draw at a lower level exceeds a power capacity of the lower level by a first amount, and a total peak power draw at a higher level exceeds a power capacity of the higher level by a second amount that is greater than the first amount.

28. The method of claim 1, wherein determining an oversubscription ratio includes calculating a ratio of peak utilization of one computer in the group of computers to a peak utilization per computer for the group of computers.