Temperature management in a data storage system

• US 10,156,987 B1
• Filed: 06/26/2015
• Issued: 12/18/2018
• Est. Priority Date: 06/26/2015
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Gutierrez, Felipe Enrique Ortega, Frink, Darin Lee
• Primary Examiner(s): Perez-Velez, Rocio Del Mar
• Application Number:
  US14/752,799
• Time in Patent Office: 1,271 Days
• CPC Class Codes: G06F 1/20 Cooling means
• CPC Class Codes: G06F 1/206 comprising thermal management
• CPC Class Codes: G06F 1/3268 Power saving in hard disk drive
• CPC Class Codes: G06F 1/3287 by switching off individual functional units in the computer system
• CPC Class Codes: G06F 11/0793 Remedial or corrective actions recovery from an exception in an instruction pipeline G06F9/3861; by retry G06F11/1402; for recovering from a failure of a protocol instance or entity H04L69/40
• CPC Class Codes: G06F 11/3034 where the computing system component is a storage system, e.g. DASD based or network based digital input from or digital output to record carriers G06F3/06; digital recording or reproducing G11B20/18; for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS], H04L67/1097
• CPC Class Codes: G06F 11/3058 Monitoring arrangements for monitoring environmental properties or parameters of the computing system or of the computing system component, e.g. monitoring of power, currents, temperature, humidity, position, vibrations thermal management in cooling arrangements of a computing system G06F1/206
• CPC Class Codes: G06F 11/3062 where the monitored property is the power consumption power management in a computing system G06F1/3203
• CPC Class Codes: G06F 2201/81 Threshold
• CPC Class Codes: G06F 3/0617 in relation to availability
• CPC Class Codes: G06F 3/0634 by changing the state or mode of one or more devices
• CPC Class Codes: G06F 3/0689 Disk arrays, e.g. RAID, JBOD
• CPC Class Codes: Y02D 10/00 Energy efficient computing, e.g. low power processors, power management or thermal management

Abstract

Techniques are described for managing temperatures within a data storage system by selectively interrupting power to one or more components of the data storage system. Temperature sensors measure the temperature of various components included in racks of a data center, such as data storage modules, backplanes of data storage modules, or mass storage devices coupled to backplanes. A control device may determine that a thermal event, such as a higher than threshold temperature, is occurring in one or more components. The control device may emit signal(s) to instruct power distribution unit(s) to selectively interrupt or reduce the power sent to those component(s) exhibiting the thermal event. The components may also be instructed to reduce a number of operations being performed. In some cases, fan speeds may be selectively adjusted to cool the component(s) and thus mitigate the thermal event. Power consumption may be employed to infer the temperature of component(s).

Claims

1. A system, comprising:

a rack;

a data storage module coupled to the rack, the data storage module comprising;

a first backplane;

at least one first mass storage device coupled to the first backplane;

at least one second backplane;

at least one second mass storage device coupled to the at least one second backplane; and

a control device configured to;

access status information indicating;

a first current temperature of at least one of the first backplane or the at least one first mass storage device;

a second current temperature of at least one of the at least one second backplane or the at least one second mass storage device;

determine that the first current temperature exceeds a first threshold temperature and that the second current temperature does not exceed the first threshold temperature;

responsive to the first current temperature exceeding the first threshold temperature, emit at least one signal to cause a shutdown of the at least one first mass storage device coupled to the first backplane while the at least one second mass storage device remains operative; and

subsequent to the shutdown of the at least one first mass storage device;

responsive to a determination that the first current temperature is greater than a second threshold temperature and less than the first threshold temperature, wherein the second threshold temperature is less than the first threshold temperature, maintain the shutdown of the at least one first mass storage device; and

responsive to a determination that the first current temperature is less than the second threshold temperature, emit at least another signal to cause a restart of the at least one first mass storage device.

2. The system of claim 1, wherein:

the rack comprises at least one power distribution unit (PDU) communicatively coupled to the control device, the at least one PDU configured to provide power to the first backplane and the second backplane;

the control device is further configured to send the at least one signal to the at least one PDU; and

the at least one PDU is further configured to, responsive to receiving the at least one signal, interrupt the power to the first backplane to cause the shutdown of the at least one first mass storage device coupled to the first backplane.

3. The system of claim 1, wherein:

the control device is communicatively coupled to the at least one first mass storage device;

the control device is further configured to send the at least one signal to the at least one first mass storage device; and

the at least one first mass storage device is configured to initiate the shutdown responsive to receiving the at least one signal.

4. A system, comprising:

one or more data storage modules configured to couple to a rack, the one or more data storage modules comprising one or more backplanes configured to couple to one or more mass storage devices; and

a control device configured to;

determine a temperature of one or more of the rack, the one or more data storage modules, the one or more backplanes, or the one or more mass storage devices;

responsive to the temperature exceeding a first threshold temperature, determine a component group comprising one or more of the rack, the one or more data storage modules, the one or more backplanes, or the one or more mass storage devices;

emit at least one signal to cause at least one action for lowering the temperature, the at least one action comprising one or more of;

a shutdown of the component group, wherein at least one of the one or more mass storage devices remains active after the shutdown of the component group;

a reduction in a number of operations performed in the component group;

a speed adjustment for at least one cooling component arranged to move air in proximity to the component group;

or

a reduction in power supplied to the component group; and

subsequent to performance of the at least one action;

responsive to a determination that the temperature is greater than a second threshold temperature and less than the first threshold temperature, wherein the second threshold temperature is less than the first threshold temperature, maintain the at least one action; and

responsive to a determination that the temperature is less than the second threshold temperature, emit at least another signal to cause at least a partial reversal of the at least one action.

5. The system of claim 4, wherein:

the component group is a first component group;

the at least one signal is at least one first signal;

the at least one action is at least one first action; and

the control device is further configured to;

responsive to the temperature continuing to exceed the first threshold temperature after the at least one first action, determine a second component group comprising more mass storage devices than the first component group; and

emit at least one second signal to cause at least one second action for lowering the temperature, the at least one second action comprising one or more of;

a shutdown of the second component group;

or

a speed adjustment for at least another cooling component arranged to move air in proximity to the second component group.

6. The system of claim 5, wherein:

the temperature is of a backplane of the one or more backplanes;

the first component group comprises a subset of the one or more mass storage devices coupled to the backplane; and

the second component group comprises the backplane.

7. The system of claim 4, wherein:

the control device is further configured to access status information describing the temperature as measured by at least one temperature sensor arranged to measure the temperature of the one or more of the one or more data storage modules, the one or more backplanes, or the one or more mass storage devices.

8. The system of claim 4, wherein the control device is further configured to:

access status information that indicates a power consumption of at least one of the one or more data storage modules, the one or more backplanes, or the one or more mass storage devices; and

determine the temperature based at least partly on the power consumption.

9. The system of claim 4, wherein the control device is further configured to calculate the temperature of a given mass storage device as at least a function of:

an ambient temperature as measured at a location, in the data storage module, separate from the given mass storage device;

a power consumption of the given mass storage device;

a location of the given mass storage device in the data storage module;

an airflow in proximity to the given mass storage device;

or

a previously calibrated thermal resistance of the given mass storage device.

10. The system of claim 4, wherein:

the control device is further configured to cause the at least one signal to be communicated to one or more of;

at least one microcontroller of at least one mass storage device included in the component group;

or

at least one control device that controls operations performed on the at least one mass storage device; and

the at least one action further comprises instructing one or more of the at least one microcontroller or the at least one control device to reduce the number of operations performed on the at least one mass storage device.

11. The system of claim 4, wherein:

the control device is further configured to send the at least one signal to a power distribution unit (PDU) that is configured to provide power to the one or more backplanes; and

the PDU is configured to, responsive to receiving the at least one signal, interrupt the power to the one or more backplanes included in the component group to cause the shutdown of the one or more mass storage devices coupled to the one or more backplanes.

12. The system of claim 4, wherein the at least one action is based, at least partly, on a difference between the temperature and the threshold temperature.

13. A method, comprising:

accessing status information indicating a temperature of one or more components of a data storage system, the one or more components including one or more of;

a mass storage device, a backplane coupled to the mass storage device, or a data storage module that includes the backplane;

based on the status information, detecting a thermal event in the data storage system, wherein the thermal event is associated with the temperature exceeding a first threshold temperature;

determining a component group comprising the at least one of the one or more components associated with the thermal event; and

emitting at least one signal to cause at least one action in response to the thermal event, the at least one action comprising one or more of;

a shutdown of the component group, wherein at least one of the one or more mass storage devices remains active after the shutdown of the component group;

a reduction in power supplied to the component group;

a reduction of activity in the component group;

or

a speed adjustment to increase a speed of at least one cooling component arranged to move air in proximity to the component group;

subsequent to causing the at least one action;

responsive to a determination that the temperature is between the first threshold temperature and a second threshold temperature, maintaining the at least one action, wherein the first threshold temperature is greater than the second threshold temperature; and

responsive to a determination that the temperature is less than the second threshold temperature, emitting at least another signal to cause at least partial reversal of the at least one action.

14. The method of claim 13, further comprising:

determining, based on the status information, an ambient temperature in proximity to the data storage system;

determining, based at least partly on the ambient temperature, a minimum speed and a maximum speed for the at least one cooling component; and

configuring the at least one signal to cause the speed adjustment within a range bounded by the minimum speed and the maximum speed.

15. The method of claim 13, further comprising:

accessing power consumption information that indicates a power consumption of at least one of the one or more components, the power consumption information received from the at least one of the one or more components; and

determining the temperature further based, at least partly, on the power consumption.

16. The method of claim 13, further comprising:

identifying data stored on the one or more mass storage devices included in the component group; and

prior to the shutdown of the component group, migrating at least a portion of the data to at least one other mass storage device.

17. The method of claim 13, wherein the thermal event is characterized by a change in the temperature during a time period exceeding a threshold change.