Process for incrementally commissioning mechanical infrastructure in a data center

• US 10,334,758 B1
• Filed: 06/05/2015
• Issued: 06/25/2019
• Est. Priority Date: 06/05/2015
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Ramirez, Antonio William Vasquez, Phillips, Matthew Thomas, Kaplan, Faran Harold
• Primary Examiner(s): Tugbang, A. Dexter
• Application Number:
  US14/732,573
• Time in Patent Office: 1,481 Days
• CPC Class Codes: G01R 31/58 Testing of lines, cables or conductors testing of electric windings G01R31/72
• CPC Class Codes: G06F 1/189 Power distribution
• CPC Class Codes: G06F 1/26 Power supply means, e.g. regulation thereof for memories G11C
• CPC Class Codes: H02J 9/06 with automatic change-over , e.g. UPS systems
• CPC Class Codes: H02J 9/061 for DC powered loads
• CPC Class Codes: H05K 7/18 Construction of rack or frame
• CPC Class Codes: H05K 7/20709 for server racks or cabinets; for data centers, e.g. 19-inch computer racks
• CPC Class Codes: H05K 7/20718 Forced ventilation of a gaseous coolant in closed loop H05K7/20754
• CPC Class Codes: H05K 7/20745 within rooms for removing heat from cabinets, e.g. by air conditioning device
• CPC Class Codes: H05K 7/20754 Air circulating in closed loop within cabinets
• CPC Class Codes: H05K 7/20836 Thermal management, e.g. server temperature control
• CPC Class Codes: H05K 7/20945 Thermal management, e.g. inverter temperature control
• CPC Class Codes: Y10T 29/49004 including measuring or testing of device or component part
• CPC Class Codes: Y10T 29/4902 Electromagnet, transformer or inductor

Abstract

Data center mechanical infrastructure is incrementally deployed and commissioned to support incremental changes in computing capacity in a data center while mitigating interaction between infrastructure being commissioned and installed computer systems. Incremental mechanical infrastructure commissioning can be concurrent with incremental electrical infrastructure commissioning and includes operating mechanical infrastructure to remove heat generated as a result of operating electrical infrastructure to support simulated electrical loads as part of electrical infrastructure commissioning. Incremental mechanical infrastructure deployment can be based on the power support capacity provided by incrementally deployed electrical infrastructure. Incremental infrastructure deployment can include partitioning a space in which incremental mechanical infrastructure is configured to provide cooling, so that heat generation and removal in the space, based on commissioning the incremental mechanical infrastructure, is isolated electrical and cooling support provided to electrical loads located in a remainder of the data center.

Claims

1. A method for incrementally commissioning mechanical infrastructure in a data center, wherein the method comprises:

installing a power distribution system in the data center, where the power distribution system is configured to provide electrical power support to a plurality of sets of rack computer systems in a plurality of computer spaces in the data center;

installing a set of power system mechanical infrastructure which is configured to provide cooling support to the power distribution system;

verifying that the set of power system mechanical infrastructure is configured to provide cooling support to the power distribution system, wherein the verifying comprises;

simulating electrical power consumption by the plurality of sets of rack computer systems, based on operation of a set of load banks electrically coupled to the power distribution system, wherein the set of load banks are test loads that simulate the electrical power consumption of one or more live loads, such that the power distribution system generates heat as a result of supporting the simulated electrical power consumption by the plurality of sets of rack computer systems; and

operating the set of power system mechanical infrastructure, concurrently with the power distribution system supporting the simulated electrical power consumption by the plurality of sets of rack computer systems, such that the set of power system mechanical infrastructure removes the heat generated by the power distribution system as a result of supporting the simulated electrical power consumption by the plurality of sets of rack computer systems;

installing a set of power distribution infrastructure in a computer space, of the plurality of computer spaces, such that the set of power distribution infrastructure is configured to supply electrical power received from the power distribution infrastructure to rack computer systems installed in the computer space;

installing a set of computer space mechanical infrastructure proximate to the computer space, wherein the set of computer space mechanical infrastructure is configured to provide cooling support to a set of rack computer systems, of the plurality of sets of rack computer systems, and power distribution infrastructure located in the computer space; and

verifying that the set of computer space mechanical infrastructure is configured to provide cooling support to a set of rack computer systems, of the plurality of sets of rack computer systems, and power distribution infrastructure located in the computer space, wherein the verifying comprises;

simulating electrical power consumption by the set of rack computer systems in the computer space, based on operation of at least one load bank electrically coupled to the set of power distribution infrastructure in the computer space, wherein the at least one load bank is at least one test load that simulates the electrical power consumption of one or more live loads, such that the power distribution infrastructure generates heat, in the computer space, as a result of supporting the simulated electrical power consumption by the set of rack computer systems; and

operating the set of computer space mechanical infrastructure, concurrently with the power distribution infrastructure supporting the simulated electrical power consumption by the set of rack computer systems in the computer space, such that the set of computer space mechanical infrastructure removes, from the computer space, the heat generated by the power distribution infrastructure as a result of supporting the simulated electrical power consumption by the set of rack computer systems.

2. The method of claim 1, wherein:

the verifying that the set of computer space mechanical infrastructure is configured to provide cooling support to the set of rack computer systems and power distribution infrastructure located in the computer space comprises partitioning the computer space from a remainder of the data center, such that;

airflow between the computer space and the remainder of the data center is restricted; and

cooling support provided to the computer space via operation of the set of computer space mechanical infrastructure is isolated from being provided to rack computer systems located in the remainder of the data center.

3. The method of claim 2, wherein:

the remainder of the data center comprises;

a deployed set of rack computer systems consuming electrical power supplied by the power distribution system, and

a deployed set of computer space mechanical infrastructure providing cooling support to the separate set of rack computer systems;

the verifying that the set of computer space mechanical infrastructure is configured to provide cooling support to the set of rack computer systems and power distribution infrastructure located in the computer space comprises;

simulating electrical power consumption by the of set of rack computer systems in the computer space in isolation from the remainder of the data center, such that heat generated in the computer space is restricted from reaching the deployed set of rack computer systems; and

the method further comprises, subsequent to verifying that the set of computer space mechanical infrastructure is configured to provide cooling support to the set of rack computer systems and power distribution infrastructure located in the computer space;

adjusting the set of computer space mechanical infrastructure to operate in unison with the deployed set of computer space mechanical infrastructure.

4. The method of claim 1, wherein the simulating electrical power consumption by the plurality of sets of rack computer systems is performed prior to installation of the plurality of sets of rack computer systems in the data center and comprises:

electrically coupling the set of load banks, in parallel, to the power distribution system;

wherein each separate load bank, of the set of load banks, is configured to simulate electrical power consumption by a separate set of rack computer systems, of the plurality of sets of rack computer systems, such that the set of load banks collectively simulate electrical power consumption by the plurality of sets of rack computer systems.

5. The method of claim 4, wherein:

the simulating electrical power consumption, by the plurality of sets of rack computer systems, comprises;

coupling an initial load bank, of the set of load banks, to an initial set of power distribution infrastructure, in an initial computer space, which is electrically coupled to the power distribution system, such that the initial set of power distribution infrastructure generates heat as a result of supporting the simulated electrical power consumption of at least one initial set of rack computer systems in the initial computer space; and

the verifying further comprises operating a set of initial computer space mechanical infrastructure, concurrently with the initial set of power distribution infrastructure supporting the simulated electrical power consumption of the at least one initial set of rack computer systems in the initial computer space, such that operating temperatures in each portion of the initial computer space remain less than threshold operating temperatures associated with a respective portion of the initial computer space.

6. The method of claim 5, comprising:

verifying that the set of power system mechanical infrastructure in the data center is configured to provide cooling support to an additional set of rack computer systems, of the plurality of sets of rack computer systems, in an additional computer space, of the plurality of computer spaces, wherein the verifying the set of power system mechanical infrastructure comprises, subsequently to installing the initial set of rack computer system in the initial computer space;

simulating electrical power consumption by the additional set of rack computer systems in the additional computer space, based on supplying electrical power through an additional set of power distribution infrastructure to an additional load bank in the additional computer space, such that an additional power busway generates heat in the additional computer space as a result of supporting the simulated electrical power consumption by the additional set of rack computer systems; and

operating an additional set of computer space mechanical infrastructure, concurrently with the additional set of power distribution infrastructure supporting the simulated electrical power consumption by the additional of set of rack computer systems, such that operating temperatures in each portion of the additional computer space remain less than threshold operating temperatures associated with a respective portion of the additional computer space.

7. The method of claim 6, wherein:

the verifying the set of power system mechanical infrastructure comprises partitioning the additional computer space from the initial computer space, such that airflow between the initial computer space and the additional computer space is restricted and the heat generated in the additional computer space is restricted from heating the initial set of rack computer systems; and

the additional set of computer space mechanical infrastructure is operated in isolation from the set of initial computer space mechanical infrastructure, such that the additional set of power distribution infrastructure is restricted to receiving cooling support from the additional set of computer space mechanical infrastructure.

8. The method of claim 7, comprising:

subsequent to the verifying that the mechanical infrastructure in the data center is configured to provide cooling support to the additional set of rack computer systems in the additional computer space;

adjusting the additional set of computer space mechanical infrastructure to operate at a common operating state with the set of initial computer space mechanical infrastructure; and

subsequent to the adjusting the additional set of computer space mechanical infrastructure, removing airflow restrictions between the initial computer space and the additional computer space.

9. The method of claim 8, wherein:

the set of computer space mechanical infrastructure comprises a set of air handling units configured to induce a cooling air flow through a separate computer space;

the data center comprises an intake air header unit which is configured to supply intake air to each computer space, of the plurality of computer spaces, in parallel; and

partitioning the additional computer space from the initial computer space comprises partitioning an interior of the intake air header unit, so that intake air supplied to a particular set of air handling units, included in the initial set of computer space mechanical infrastructure, is isolated from being supplied to a separate set of air handling units included in the additional set of computer space mechanical infrastructure.

10. The method of claim 6, wherein:

the set of power system mechanical infrastructure in the data center is configured to remove at least a particular amount of heat from the data center; and

the method comprises, prior to verifying that the set of power system mechanical infrastructure in the data center is configured to provide cooling support to the additional set of rack computer systems;

determining that an additional set of power distribution infrastructure which is configured to supply a particular amount of electrical power is installed in the data center;

determining supplying the particular amount of electrical power is associated with generation of a particular amount of heat in the data center; and

installing the additional set of computer space mechanical infrastructure in the data center, based on a determination that the additional set of computer space mechanical infrastructure is configured to provide sufficient cooling support to cause the particular amount of heat to be removed from the data center.

11. The method of claim 1, wherein:

the set of computer space mechanical infrastructure comprises one or more of;

a set of direct evaporation air cooling units configured to induce cooling of an air flow;

a set of active air cooling units configured to induce cooling of an air flow via heat transfer with a circulating coolant;

a set of mechanical cooling units configured to induce cooling of an air flow via operation of a mechanical device;

or

a set of exhaust air moving devices configured to induce an exhaust airflow from a given space.

12. The method of claim 1, wherein:

the verifying that the set of computer space mechanical infrastructure is configured to provide cooling support to the set of rack computer systems comprises isolating an individual computer space from a remainder of computer spaces, of the plurality of computer spaces, prior to simulating electrical power consumption by a particular set of rack computer systems in the individual computer space, such that;

airflow between the individual computer space and the remainder of computer spaces is restricted concurrently with the simulating electrical power consumption by the particular set of rack computer systems; and

the quantity of heat generated in the individual computer space is restricted from heating separate sets of rack computer systems located in the remainder of computer spaces.

13. The method of claim 12, wherein operating the set of computer space mechanical infrastructure comprises:

operating the set of computer space mechanical infrastructure in isolation from operation of separate sets of computer space mechanical infrastructure which remove heat from the separate sets of rack computer systems in the remainder of computer spaces, such that heat generated by the separate sets of rack computer systems is removed independently of operation of the set of computer space mechanical infrastructure.

14. The method of claim 13, comprising:

subsequent to verifying that the set of computer space mechanical infrastructure in the data center is configured to support cooling of the set of rack computer systems in the individual computer space;

adjusting the set of computer space mechanical infrastructure to operate at a common operating state with the separate sets of computer space mechanical infrastructure; and

subsequent to the adjusting the set of computer space mechanical infrastructure, removing airflow restrictions between the individual computer space and the remainder of computer spaces.

15. The method of claim 12, wherein:

the set of computer space mechanical infrastructure in the data center comprises a set of air handling units configured to induce a cooling air flow through a separate computer space;

the data center comprises an intake air header unit which is configured to supply intake air to each computer space, of the plurality of computer spaces, in parallel; and

isolating the individual computer space from the remainder of computer spaces comprises partitioning an interior of the intake air header unit, so that intake air supplied to a particular set of air handling units, included in the set of computer space mechanical infrastructure, is isolated from being supplied to a separate set of air handling units included in the separate sets of computer space mechanical infrastructure.

16. The method of claim 12, wherein the simulating electrical power consumption by the particular set of rack computer systems in the individual computer space comprises:

the set of power distribution infrastructure comprises a power busway which extends through the individual computer space and which is configured to be electrically coupled, at one end, to a power distribution system; and

simulating electrical power consumption by the particular set of rack computer systems in the individual computer space comprises coupling the load bank located in the individual computer space to an opposite end of the power busway, such that the power busway supplies electrical power received from the power distribution system to the load bank located in the individual computer space via an entirety of the power busway.

17. The method of claim 16, wherein:

the power distribution system is configured to supply electrical power to the separate sets of computer systems, in the remainder of computer spaces, in parallel with the power busway extending through the individual computer space; and

the method comprises, prior to verifying that the set of computer space mechanical infrastructure in the data center is configured to support cooling of the set of rack computer systems in the individual computer space, verifying that the set of computer space mechanical infrastructure in the data center is configured to support cooling of at least the power distribution system, wherein the method further comprises;

simulating electrical power consumption, by at least the particular set of rack computer systems and the separate sets of rack computer systems, of electrical power supplied by the power distribution system, such that the power distribution system generates another quantity of heat as a result of supporting the simulated electrical power consumption by at least the particular set of rack computer systems and the separate sets of rack computer systems; and

operating the set of power system mechanical infrastructure, concurrently with the power distribution system supporting the simulated electrical power consumption by at least the particular set of rack computer systems and the separate sets of rack computer systems, such that the set of power system mechanical infrastructure removes the another quantity of heat from the power distribution system.