Method and apparatus to monitor stress conditions in a system

• US 20070089011A1
• Filed: 09/26/2005
• Published: 04/19/2007
• Est. Priority Date: 09/26/2005
• Status: Active Grant

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Mukherjee, Udayan, Wei, Wen, Hiremath, Chetan, Chandwani, Neelam, Dodeja, Rakesh
• Granted Patent: US 7,424,396 B2
• Time in Patent Office: Days
• US Class Current: 714/742
• CPC Class Codes: G05B 23/0235 based on a comparison with predetermined threshold or range, e.g. "classical methods", carried out during normal operation; threshold adaptation or choice; when or how to compare with the threshold
• CPC Class Codes: G06F 11/3044 where the computing system component is the mechanical casing of the computing system
• CPC Class Codes: G06F 11/3058 Monitoring arrangements for monitoring environmental properties or parameters of the computing system or of the computing system component, e.g. monitoring of power, currents, temperature, humidity, position, vibrations thermal management in cooling arrangements of a computing system G06F1/206

Abstract

Faults are monitored with information from agents for a plurality of sensors located on a plurality of circuit boards. A policy containing a error event thresholds against which the stored sensor information can be compared. Actions can be initiated by a fault module when one or more of the error event thresholds is exceeded.

Claims

1. A method of monitoring faults, comprising:

storing information from a plurality of agents for a plurality of sensors located on a plurality of circuit boards;

storing a policy containing a plurality of error event thresholds against which the stored sensor information can be compared; and

initiating an action, by a fault module, when one or more of the error event thresholds is exceeded.

2. The method according to claim 1, further including predicting a device failure based upon the sensor information.

3. The method according to claim 1, further including storing temperature and voltage information for a plurality of integrated circuits and counting events in which specified temperature and/or voltage parameters are exceeded.

4. The method according to claim 3, further including storing frequency information and counting events in which specified temperature, voltage and/or frequency parameters are exceeded.

5. The method according to claim 4, further including predicting failures based upon the counted events.

6. The method according to claim 5, further including predicting failures based upon a leaky bucket counter.

7. The method according to claim 4, further including predicting failures based upon a rate that the counted events accumulate.

8. The method according to claim 1, wherein the action includes generating an operator alarm.

9. The method according to claim 1, further including monitoring sensors for a chassis cooling fan.

10. The method according to claim 9, wherein the fan sensors include at least three of fan current, fan speed monitor, intake and outtake air temperature, fan motor temperature, intake and outtake differential, noise level, vibration.

11. The method according to claim 10, further including measure fan power over time.

12. The method according to claim 10, further including monitoring three or more of fan spin up time, start/stop count, power on hour count, power cycles, power consumed, temperature, spin high current, spin noise, spin speed, pressure differential.

13. A system, comprising:

a plurality of sensors for collecting information for a plurality of operating parameters;

a database to store the sensor information; and

a fault module to examine the sensor information and count events in which specified operating parameters are exceeded to predict device failures.

14. The system according to claim 13, further including a chassis to contain a plurality of blades.

15. The system according to claim 13, wherein the specified operating parameters include temperature and voltage.

16. The system according to claim 15, wherein certain ones of the temperature and voltage sensors are located proximate integrated circuits.

17. The system according to claim 13, wherein the fault module includes a leaky bucket counter to predict failures.

18. The system according to claim 13, wherein the fault module is implemented as a set of native instruction set on the processor optimized for implementing failure prediction mechanisms.

19. The system according to claim 13, wherein the fault module is implemented as a hardware functional block in an application specific integrated circuit.

20. The system according to claim 13, wherein the fault module is implemented as a hardware functional block instantiated in a processor core.

21. The system according to claim 13, wherein the fault module is implemented as a hardware functional block instantiated in a memory controller chipset.

22. The system according to claim 13, wherein the fault module is implemented as combination of hardware functional blocks instantiated in a processor core and chipset.

23. The system according to claim 13, wherein the fault module is implemented as a hardware functional block in a field programmable gate array.

24. The system according to claim 13, wherein the fault module is implemented on a dedicated core in a many-core or a multi-core processor architecture.

25. The system according to claim 13, wherein the plurality of sensors includes fan sensors to predict failure of a cooling fan.

26. The system according to claim 25, wherein the fan sensors include at least three of fan current, fan speed monitor, intake and outtake air temperature, fan motor temperature, intake and outtake differential, noise level, and vibration.

27. An article, comprising:

a storage medium having stored thereon instructions that when executed by a machine result in the following;

storing information from a plurality of agents for a plurality of sensors located on a plurality of circuit boards;

storing a policy containing a plurality of error event thresholds against which the stored sensor information can be compared;

initiating an action, by a fault module, when one or more of the error event thresholds is exceeded.

28. The article according to claim 27, further including instructions for predicting a device failure based upon the sensor information.

29. The article according to claim 27, further including instructions for storing temperature and voltage information for a plurality of integrated circuits and counting events in which specified temperature and/or voltage parameters are exceeded.

30. The article according to claim 29, further including instructions for storing frequency information and counting events in which specified temperature, voltage and/or frequency parameters are exceeded.

31. The article according to claim 30, further including instructions for predicting failures based upon the counted events.